r/todayilearned • • 8h ago

TIL a finance worker at a multinational firm was tricked into transferring $25 million to scammers after joining a video conference call where every single other participant—including the company's CFO and several of his recognized coworkers—was an AI-generated deepfake.

https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk
18.0k Upvotes

543 comments sorted by

1.9k

u/fanfanye 8h ago edited 7h ago

Feel like its a problem with the firm then when one person can transfer 25million to a random account without atleast another approval

79

u/-Poultrygeist- 7h ago

Must be a firm run by MightyMikes dad

→ More replies (1)

32

u/TempAirman2004 6h ago

Basic dual authorization and callback verification should be mandatory for eight-figure sums. No single employee should have the clearance to wire 25 million based purely on verbal instructions from a video call, real or fake.

242

u/oshinbruce 7h ago

Yeah bit of a flaw there, but in big companies moving big amounts money around is relatively common and it seems the current system allows one person to hit the send button in the end

306

u/maverick4002 7h ago

Absolutely not. I work in a big firm and there is no way that one person can transfer $25M dollars without further approval needed.

1000% a flaw in the process

53

u/Secret_Side-ofJ 6h ago

Depends entirely on your company. My current company that could absolutely happen, and I was regularly spending 15 million+ without any prior approvals. Depends entirely on the revenue turn of your business.

121

u/SimmeringGiblets 6h ago

Hey, it's me, your CFO, can you hop on a zoom call real quick?

55

u/Narradisall 5h ago

To me that just sounds like poor financial controls.

In any company I’ve worked money transfers of any level require at least two authorisers and would be dependent on their seniority for the level of value transferred.

Not that it doesn’t happen in businesses often, but usually because they lack decent financial controls, not because of how much revenue they turnover.

25

u/shadowfaxbinky 3h ago

The only place I’ve worked where I could transfer such large sums without a secondary authoriser was a place where it turned out the head of finance was embezzling. So yeah, very deliberate use of poor financial controls (in other areas too) to enable her fraud.

→ More replies (1)

25

u/xsvfan 6h ago

It's more private vs public. Public companies will be required from their auditors to have multiple approvals, even if it's just manager plus designated approval level.

14

u/meeu 6h ago

Do those approvals have to be authenticated in some way, or would for instance a video call with several superiors giving verbal approval be sufficient?

23

u/ChancelorReed 6h ago

No, the approvals need to be an actual sign off in the system. Someone has to log into the banking portal to push it to the next step in the process. No one person can prep and send money of typically any amount (that's a classic separation of duties issue) and then larger amounts need dual approvals after a preparation step.

7

u/DemosthenesOrNah 6h ago

when I was doing audit prep our signatures were double layered. one person signed a word doc and then someone above that signed (and locked) a PDF version of the docx. stuff doesn't move forward until the double signed pdf is handed over.

so in this case no the verbal wouldn't have sufficed, they would've needed to forward around the word doc, get it signed, forward that up a step and get the doc reviewed, pdfd, and signed again

→ More replies (1)

11

u/Trust_Me_Im_a_Panda 3h ago

No, it doesn’t depend on the revenue turn of your business, it depends on how structured and wise your company’s policies and procedures are. There’s not a single company in the world for whom $25m is “chump change” and any company who would allow a $25m transfer with a single point of failure has more money than they do intelligence. That absolutely should not happen and the fact that it can and does happen does not defeat the argument that it should not.

3

u/mxzf 2h ago

Depends entirely on your company. My current company that could absolutely happen

Yeah, that's a flaw in the process. With that kind of money moving around, there absolutely should be some kind of secondary signoff. Sure it's gonna slow stuff down some, but it is a flawed process if one person can move around millions on a whim. If nothing else, it's far too tempting a target for embezzling money, even absent any scams.

6

u/slabby 4h ago

Your company's internal auditors must be asleep at the wheel. That's a major internal control issue

→ More replies (4)
→ More replies (2)

21

u/Draaly 4h ago

I approved a bill larger than that this week and the money was sent with only 3 sets of eyes on it because it was a regular expense we have

53

u/maverick4002 4h ago

Yes, so you agree. You didnt unilaterally send the bill. You did. And then 2 others did, at least. And the two others probably wasnt a video call, was it? And it was also a regular expense which is differnet from a random one off shady request.

17

u/Draaly 4h ago

bill came in. Coder sent it to me for approval. I told the coder how to categorize it, signed the invoice, and sent to AP who sent the money to the company on the bill. As far as AP is concerned, I just told them to send money and they did. This is a very normal workflow for even extremely large bills (so long as they are expected) and I work at a tier 1 financial institution. In the example in the article, someone a few tiers above even me called AP and told them to send money, so they did. Tons and tons of controls are put in place to try and avoid getting scammed, but those controls only exist because it happened to someone important at some point.

20

u/maverick4002 4h ago

Yes, and thats similar to my workload. The coder didnt just do the transaction. Or someone in AP didnt just go ahead without having other eyes on it and documenting it.

And also, big difference here is that this was a recurring expense.

I think we are on the same page here

→ More replies (1)
→ More replies (2)

11

u/MrDLTE3 4h ago

Yeah and where the fuck was $25m going into anyway? Any accounts receiving a sudden SWIFT $25m at once is an instant system flag. Like my company spends millions on procurement too but it requires paperwork like CIF and shit.

This guy seemly presses a bunch of buttons/clicks and transfers full sum $25m in a single session? The fuck.

→ More replies (2)
→ More replies (3)

99

u/timmystwin 7h ago edited 7h ago

Auditor here.

If this was possible I'd be ripping them to shreds in management letter points.

There is no fucking way this level of money should move without at least another check/signature. Because this is what happens. Things need to be fail safe, not fail deadly.

15

u/oshinbruce 7h ago

I mean sure, but is that just an esigned signature they forged too? These had the cfo and other people there, that could count as approval. Typically they are applying alot of pressure, these scenarios arent uncommon. Obviously in the 21st century it should be 2fa and all of that, but from my understanding the right person can just put in a bank account, an amount and hit send ?

→ More replies (6)
→ More replies (17)

11

u/Mosh00Rider 7h ago

I have worked at huge companies that required multiple levels of approval for my travel stipend.

10

u/chris_ut 6h ago

I’ve never worked anywhere where the same person could both initiate a wire and approve the wire

→ More replies (1)

8

u/Igoos99 4h ago

Umm, no. My company routinely moved large sums of money around. There were all sorts of approvals required. Most routine stuff moves pretty rapidly because we are familiar with the names and contracts (even the giant amounts,) but anything new different that goes through to an unfamiliar payee gets an extra look.

→ More replies (2)

17

u/MalaysiaTeacher 7h ago

Meanwhile a bank won't give you £5000 of your own money in cash without a prior appointment and copious paperwork

21

u/Jason207 6h ago

Half of that is government money laundering rules and the other half is that we literally have to order that much cash from our central vault or the government.

Back in the 90s we'd get a couple of million dollars in cash every Friday, today I doubt banks even get weekly cash deliveries.

12

u/Lermanberry 4h ago

The third half is that only a specific type of person likes to walk out with that much cash, and then they get angry at the bank when it turns out they were scammed out of it.

→ More replies (2)
→ More replies (1)

4

u/Sustainable_Twat 6h ago

To be fair, as far that as that guy was concerned, the among others, the CFO was on the call.

→ More replies (16)

5.7k

u/Arthur233 8h ago

Woah. This is going to fuel so many paranoid schizophrenia people. People now have a legitimate reason to doubt the people they are interacting with are real.

Quick edit: Wait... The article says it was 2.5 years ago. Deep fakes were obvious as hell back then. This person was put in charge of too much money and power for the level of critical thinking skills they had!

158

u/ChancelorReed 6h ago

The failure here doesn't even have anything to do with the deepfake really. The fact that any single worker could release $25m in funds without anyone else going into the banking portal is a massive control failure. No one should have that type of access in the first place.

→ More replies (6)

230

u/TheSpiralTap 7h ago

Already happening. I'm going to keep things a little vague so I don't get fired. I'm a trainer/supervisor for a call center for a utility company. If you have a home in that portion of the us, you have no choice but to use this company.

I was sitting with a new hire as they took calls. This AI "person" called in and it sounded exactly like a person. It didn't hit an uncanny valley for like 5 minutes. The robot wanted the person's account number and password. The new hire kept telling it no, so it kept repeating in a loop and that's what tipped us off this isn't just a scammer, it's AI. This thing even asked for a supervisor.

74

u/Bytewave 5h ago

This thing even asked for a supervisor.

That might have worked at the telco I used to work for. The "supervisors" were remarkably dumber than the union employees (since they were picked on the basis of keeping costs low) and sometimes agreed to completely stupid things to placate a complaining customer, it caused trouble several times.

39

u/DontAbideMendacity 4h ago

My credit card was frozen for "suspicion of fraudulent behavior"... I had tried to by lunch. I called, talked to AI for a bit, finally got a human and they unfroze my card. Until dinner, because apparently eating while on vacation is suspicious activity. This time the humans claimed that AI had flagged it and I needed to send a picture of my passport or U.S. government issued driver's license (not a thing) to unfreeze it, they had no authority to override the AI! So I cancelled my card on the spot.

20

u/Mertoot 3h ago

they had no authority to override the AI!

What.

4

u/MirroredReality 1h ago

Thank you for banking with us, the Borant Corporation. Have a lovely day :)

→ More replies (1)
→ More replies (1)

29

u/MelonElbows 3h ago

Our company has our IT send out fake emails to employees randomly to test if they will click the link. We're supposed to report any emails we suspect are fake and so far, they haven't gotten me. But I talked to someone in IT and they said so many people fail the test. This is why I have to spend an hour doing a cyber security training every year....

→ More replies (2)

1.0k

u/Neebez 8h ago edited 6h ago

Right? How do you truly verify identity that's scam proof?

Edit: 2.5 years ago you could still come up with highly convincing deep fakes had you the time, money and resources. Not to mention think about all the tech that's kept from us. They say the public is at least 10 years behind on tech.

682

u/gandraw 8h ago

"Thank you for the video call, now I know what the incoming request will be about. But as you know, all transfers over 10k have to be entered into SAP and transfers over 1m need three signatures, so John could you please input the request and then Claudia and Peter will electronically sign it, and I'll initiate the transfer as soon as I get that"

314

u/AstraJin 7h ago

Exactly, we have to jump through many many hoops to get 15k through SAP , never mind 25m!

240

u/Klldarkness 7h ago

I'm the Treasury admin at my workplace, and can transfer any amount of money, but it passes through at least 3 other people before it's released. Same procedure, no matter the amount.

  1. Requester(Accountant, VP, CEO, Etc)

  2. Me(fills out form, attaches backup)

  3. Emailed to CEO for approval. Not a reply, fresh email.

  4. Once approved, entered into the bank, cover sheet generated, combined with the wire, email approval, and back up...emailed to Director of Accounting and Accounting Manager for final review, and release.

THEN, if the amount is over $5m, our bank calls me, the VP of accounting, and sometimes the Director of accounting, to confirm the release before it exits the queue.

No two steps in a row can be the same person, and the requestor, approver, and releaser can't be the same person. It has to be seen and approved by at least 3 separate people. Despite this, most wires process in 1 hour or less.

We've built this process to be as fraud proof as possible, and still review it twice a year to see if there are any way to shore it up.

55

u/Glider103 6h ago

How many people do you think are actually fully reviewing the information vs just signing off?

At my job my boss has missed some of my wire mistakes and sometimes it is either caught by me or by accounting when they reconcile the GL.

but at that point it would be harder to get funds back from a scammer (vs in my case its a legitimate wire just the wrong amount - never had a wrong account issue fingers crossed)

32

u/Klldarkness 6h ago

Out of the 3, probably at least 2. We do catch mistakes when they happen, even if it's just a leading zero in the ABA not being present because our template is in excel and excel hates keeping our leading zero settings. Lmao.

I review at least twice, when I make it and when I enter into the bank and attach the cover sheet, and our director is very anal about reviewing before releasing as then HER job would be on the line.

We also have outgoing ACH disabled on our bank accounts, as most fraud attempts are through ACH. We are strictly fedwire for outgoing wired.

12

u/mu_zuh_dell 6h ago

Yeah at my job we have a lot of processes that have a lot of reviewers, but what's happened is my department, which does most of the final checks, has become solely responsible for any mistakes. So when I get a document, I'm not just reviewing my part, I'm reviewing the whole thing lol

9

u/Glider103 6h ago

Ugh I hate that for you

Like how come I'm to blame when 3 other people ALSO looked at this

But technically yes I am supposed to review it and if I missed it that means I didn't do MY job

3

u/mxzf 3h ago

How many people do you think are actually fully reviewing the information vs just signing off?

More than if it's just one person checking things.

You don't add layers of checks like that for perfection, you add it so that things require multiple sniff-tests by different people before they get finalized.

26

u/el_smurfo 6h ago

A PO for 25 bucks takes me a week at my employer. There are at least four approvers and the actual paperwork is generated somewhere else.

37

u/chris_ut 6h ago

There are guard rails and then there’s making your company so inefficient you can’t accomplish anything

16

u/sobrique 5h ago

And sometimes too many guardrails causes approval fatigue, and so the scrutiny is less than it should be anyway.

→ More replies (1)
→ More replies (4)
→ More replies (4)

31

u/skucera 7h ago

Spending 5m takes the signature of our freaking CEO; I have no idea how a company could have such a significant single point of failure.

25

u/samgarita 7h ago

Congratulations Gregg!

You are Single Point of Failure of the Year!

7

u/ukexpat 6h ago

No segregation of duties, lack of proper internal controls.

→ More replies (2)

6

u/1CEninja 6h ago

My bank is pretty jumpy too. Even a modest transfer to an external account that's out of the ordinary can result in me getting a call because the transaction was put on pended and can be a pain to get pushed through.

And that's without the layers of protections commercial customers have in place that little retail clients like myself don't even know about lol.

3

u/mjacksongt 5h ago

Especially since it almost certainly required vendor setup and banking details being added.

This is hard to do.

3

u/ip-cx 6h ago

Yeah but you are only as hindered as your program makes you.

Our ERP software is 20 (!) years old, there are no things like that implemented. It's all up to finance to make trustworthy payments.

Good thing I don't work in finance lol

→ More replies (1)

9

u/awmaleg 7h ago

And fill out this horrible Word doc (why is it in word still?!?)

5

u/ChancelorReed 6h ago

Yea exactly, but on top of that it shouldn't even need the guy to say that. It should be entirely enforced in the system and impossible to send the funds without those approvals.

→ More replies (1)

87

u/Pjolterbeist 7h ago

Pretty simple. Contact them back through a different channel on a different device. If you think something is out of the ordinary on your video call on you laptop, call them from your phone to verify. This works against almost any reasonable attack, from phish mail to more complex attacks like this. Scammers will use fake urgency to trick you into rash decisions, just take your time and verify.

If you are being attacked by someone with near infinite resources like the Mossad or FSB or CIA, then this might not work, but it will counter almost any attack from criminal actors.

21

u/RealMccoy13x 6h ago

I think this example is more on an edge case since even by this article, it was out of the norm to step onto a video call. I do handle BEC (business email compromise) fraud similar to this scenario and seen losses higher than $25MM. That urgency is spot on for most fraud/scam types.

The common trait is that threat actors usually have compromised the businesse mail/communication systems and the business is completely unaware. They learn the Org chart with strong emphasis on the wire initiator to the Commercial platform, and the individual who sends the wire requests to the initiator. It may be different roles dependent on company. I have seen the wire initiator or the person who has access be the CFO/CEO, and I have seen it be a regular receptionist. The fraud happens when they forge an email with fictitious wire instructions from the requestors email internal to their company. From the bank's perspective it is legitimate wire that will pass authentication/tokens/OTP/voice auth.

Rules may pick up on these instances BUT it is dependent on the business cooperation. Some will double check and thank you for stopping a multi million dollar loss others will get frustrated at you second guessing and holding their wire. We hear from them in two weeks frantically.

8

u/sobrique 5h ago

Yeah, this. The process can be solid, but you need the 'senior people' to buy into it, and never try to sidestep or give someone a hard time for following the rules. There's genuinely very few real transactions that are so urgent that they cannot wait to be verified anyway.

Which isn't always what happens, because clearly they're Important (tm) and want it to happen RIGHT NOW.

→ More replies (1)
→ More replies (1)
→ More replies (3)

26

u/JustinMcSlappy 6h ago

I work for the government, in an AI field. I assure you, we aren't ten years ahead.

8

u/stellaluna92 4h ago

I work for the government. If anything we're behind lol. I still have to use msdos every day for christssake

3

u/D_Ethan_Bones 4h ago

Bureaucratic systems can be so paranoid about new rollouts that they've still got greenscale CRTs and dot matrix printers with the perforated paper.

I actually used a greenscale CRT touchscreen in the old days, it was a xerox copier.

→ More replies (1)

4

u/boogermike 5h ago

This sounds correct and besides Justin McSlappy is a name you can trust.

→ More replies (2)

18

u/Named_after_color 4h ago

"The public is at least 10 years behind on the tech" Is not true. Public software tech is nearly at parity with the stuff that's being developed. It's mostly a matter of scale when AI is involved, not specific training methods/individual neural engineering. The latter is important to be sure, and bleeding edge stuff isn't released to the public immediately, but that's more quality control than some idea that it's too dangerous to be released or whatever.

There would be no company that wouldn't jump to release 10 years of improvement on current systems.

43

u/hydrohorton 8h ago

Mother's maiden name and first pet

27

u/Confident-Poetry6985 8h ago

Lmao, the street you grew up on, your first car.

3

u/spiritofniter 7h ago

Where did your parents meet?

19

u/Background_Honey9141 7h ago

My parents met in school. They loved being homeschooled.

3

u/what_is_blue 7h ago

I’m still trying to remember my “Favourite food” to get into my old hotmail and grab some photos from my grandpa’s 70th.

Can’t reset via email because the other email address is with AOL and it’s long since expired.

However, I can’t set up a new one with the same address because I might be able to… hack myself?

3

u/Confident-Poetry6985 7h ago

Pretty sure a lot of them got deleted. All my shit was gone when I finally got back in if I remember correctly.

→ More replies (1)
→ More replies (3)

4

u/SoRedditHasAnAppNow 7h ago

I'm confused, why would anyone want to know Mercy Blvd and Toyota Tercel?

5

u/MakingItElsewhere 7h ago

What? I think you mean Joy and Mustang!

→ More replies (3)
→ More replies (3)

24

u/5litergasbubble 7h ago

Ive told my mom to ask me how my dog is if she ever gets a call claiming to be me. Since he died a few years ago the response should make it clear if its a scam or not

33

u/Cultural-Company282 7h ago

"Wolfie is just fine, dear."

9

u/ddr4memory 6h ago

That's what I thought of.

"Your fosta parents are dead"

21

u/spacestonkz 7h ago

I told my parents if they're in doubt to ask if I'd heard from my sister lately.

I don't have a sister. My dad caught a bank scammer pretending to be me.

He's hard of hearing so a crying female voice sounds about the same for everyone to him. He thought it was weird I called him and not my not my mother who hears like an eagle (they are retired and always together). He said he asked "did you talk to your sister yet?" And the scammer said "yes, but ...". So he hung up and blocked the number.

11

u/TiddiesAnonymous 7h ago

Bruh

My dad picked family pets name for the phone account

I said huh? We never really had pets. We had a goldfish named Dave. My sister has a dog named Carmela, it must be that. No dice. Tried for 10 minutes naming any pet I knew.

Called back and got a different person, "oh ok. It says 'NONE' " 🤬

3

u/lekker-boterham 6h ago

This is hilarious

6

u/_CMDR_ 7h ago

You have just burned that identification from posting it on reddit. Get a new one.

→ More replies (1)
→ More replies (2)

3

u/SoRedditHasAnAppNow 7h ago

What does Thompson and Fuzzles have to do with this?

→ More replies (6)

12

u/ZirePhiinix 7h ago

You call them back using your own channels. It's not that hard to establish SOP.

11

u/Igoos99 4h ago

We’re taught to verify every request. Hang up, search for the requester’s number in your internal system and call them to verify.

Every last high level management person made it absolutely clear they were happy to be called and happy to be verified no matter how urgent the matter.

Plus, releasing any significant sum of money required more than one level of signature/approval before it would be released. No one person no matter how high or low on the totem pole could release that sum of money solo.

The blame for something like this should never reside on a single person.

6

u/Xenon009 4h ago

Easy to say that until the "Boss" is shouting at you to do it right now, or we'll lose the client, and your job is on the line. People are pretty fuckin smart when they're calm, once someone starts shouting and panicking, it gets very dumb and very contagious pretty fast.

6

u/Igoos99 4h ago

That exactly why that “boss” is part of the training where they say it’s okay to double check them. If you have good systems in place, stuff like this doesn’t happen. If you have abusive bosses that will yell at you for not recklessly transferring millions of dollars without observing safety procedures, then that is the boss’s fault not the employee.

6

u/ruffledcolonialgarb 3h ago

This is why training to strict procedure is so essential. Route memory kicks in, plus policy protects the employee. It does rely on top down buy in, though. 

3

u/goat_penis_souffle 4h ago

Ive been on enough client engagements for controls reviews (usually after a big loss or audit/regulatory finding) to see that many institutions have a callback procedure for client dispersements/external payments that involve calling the non-requesting individual on a known-good number. Trouble is, if you’re dealing with a security breach they’ve probably got control of your PBX too.

13

u/FacetiousTomato 7h ago

The government is nowhere close to 10 years ahead of us on this. The companies pouring hundreds of billions into this stuff are naturally way way further ahead. They might give governments preferential access to things, but they're not holding back on what they're selling, or it would hurt the bottom line. And they won't do that.

→ More replies (8)

7

u/ProletarianLilith 6h ago

Naw this dude is just dumb

6

u/ChancelorReed 6h ago

You make a process where it's impossible for a single individual to release funds on their own. That's the actual problem here.

A well controlled (and completely standard) process involves one person preparing a transaction, a separate person reviewing and approving it before release, and then on top of that would have dollar value thresholds where someone else (like the CFO) would have to log in and release it themselves if it was large enough.

→ More replies (1)

5

u/sobrique 6h ago

I 'deepfaked' myself as part of a security demonstration. It's honestly shockingly easy now.

Mine was a bit crude - I gave an LLM a few photos, and said 'create a video of this saying ...,.'

And it was plausible - I mean, no one on that room was fooled, because it clearly wasn't my accent, but still. "just" a photo was good enough for a short clip.

Video calls have the 'benefit' of being potentially poor quality lines, so you don't need to do real time hires or anything, and voice quality being 'off' isn't as apparent.

But ultimately the 'verification' has to be done on a web of trust basis - don't assume someone else calling you is legit. Don't assume someone else setting up the video call is legit.

You can set up email to have high levels of verification - if it's entirely internal, you can reasonably reliably assert that that email was sent from that person's account at least.

Externally encryption still works the way it always used to, but lots of people still can't be bothered with the hassle of key exchange and validation needed to do PGP or similar.

But you can do 'identity management' services, a bit like you'll probably have encounterd with 'google login' or 'twitter auth' or similar - companies can do their own, which means you can set rules around needing an access token, a biometric, a password, encryption certificates etc.

If it's someone you know then calling them using a phone number you look up on the company address book could likewise be 'reasonable' validation.

But honestly this was 2.5 years ago. I'm absolutely certain that the state of deepfakes is now better than ever, cheaper than ever, and can probably be automated with an LLM behind it.

You can have a conversation with most of them if you put them into 'voice mode' and it'd be hard to tell they weren't a real person. Especially if you prompt them with 'reasonable excuses' like "sorry, I'm in a bad signal area" or "I'm in a hurry for (fabricated reason)", so really you have to be hard and firm with 'follow the procedure, do not EVER rush even if the person calling you is authoritative and in a panic.

4

u/pakman82 3h ago

i work in tech. I know fintech processes that send multi 20+mil transactions daily, with boiler plate signatures & automated acceptance & a 60k /yr peon validating them..

11

u/Accidental-Genius 7h ago

Authentication codes.

Or ya know just regular due diligence.

“Hey good morning, before we kick off how was little Jimmy’s T-Ball game the other night?”

If it’s a scam the CEO will confidently answer. If it’s not, the CEO will be confused because he/she doesn’t have a kid named Jimmy.

→ More replies (1)

3

u/leftofdanzig 7h ago

“Great, can you create a ticket and CC your manager with the request? I’ll send it up the chain for approval.”

5

u/Greedyanda 7h ago

They say the government is at least 10 years ahead of us.

Maybe in very specific sectors like hypersonic aviation, radar, and cryptography but sure as hell not overall and certainly not in AI.

→ More replies (29)

86

u/dashingThroughSnow12 7h ago

There was a range to deepfakes 2.5 years ago. The quick, cheap ones were obvious. If you had money and time to blow to train one, it could be convincing.

The revolution over the past couple of years has been to narrow the gap between the two.

12

u/BulbusDumbledork 7h ago

you'd still need hundreds of hours of footage for each person in order to train the fakes. you can drop it down to dozens of hours each if you have zoom footage from fixed angles and lighting and you're just reproducing that

but you'd still need a lot of post processing vfx to make it convincing. even disney wasn't hitting out the park. and that's on offline footage. doing it live is a tall order

there was a lot of social engineering and idiocy involved for this to work with deepfakes.

deepfakes have kinda fallen on the wayside as ai generated style transfer has gotten much better (deepfakes modify existing pixels with generated ones, style transfer creates all new pixels from scratch)

22

u/Lawlcopt0r 7h ago

I'm not a scammer but I assume they made it as good as they could and then threw a whole bunch of low rez webcam distortion on top of it to mask any inconsistencies. You don't expect everyone thst videocalls you to have a perfect internet connection

11

u/mightylordredbeard 6h ago

I mean you can just lower the video quality a bit, have most of the “people” not move around or talk too much, and maybe have them situated further away from the camera. Then it’d probably fool anyone that isn’t too tech literate.

→ More replies (2)

27

u/UKS1977 8h ago

It was an excuse....

70

u/fer_sure 7h ago

Deep fakes were obvious as hell back then.

Not on a 17" laptop screen, in a Zoom call with several attendees. Each deepfake wouldn't stand up to scrutiny, but they're only an couple of inches high. And the "my voice sounds different because of a bad connection" ploy is as old as the first prank phone call.

The employee probably bypassed some kind of safeguards to transfer the money, but you can see why they would.

5

u/iconocrastinaor 2h ago

If I recall correctly this person was called into a conference room at his office where everybody else also conferenced in remotely. That actually added to the legitimacy of the request in his mind. Also this took place in Asia where deference to authority and seniority counts a whole lot more than in America.

6

u/YouToot 6h ago

Exactly.

It's funny, everyone seems to think they can spot this stuff and you must be an idiot to get tricked.

Going forward it will be impossible to tell the difference so enjoy feeling superior for like another 6 months but in the end everyone will be getting tricked.

→ More replies (1)

10

u/Yglorba 4h ago

Quick edit: Wait... The article says it was 2.5 years ago. Deep fakes were obvious as hell back then. This person was put in charge of too much money and power for the level of critical thinking skills they had!

Eh. I feel like 2.5 years ago was the precise point where they existed for people who were willing to look for and spend money on them, and most people didn't know that convincing real-time deepfakes existed and therefore wouldn't be on guard for them.

33

u/mightylordredbeard 6h ago

You misunderstand. The deepfakes that the public had access to weren’t that great, but the ones that certain governments had access to were better than what we currently have. That’s why many believe this wasn’t just random scammers, but someone/people with the backing of a powerful nation or someone who managed to get their hands on that technology.

8

u/chux4w 7h ago

Worker: You want me to transfer $25 million?

Boss: Yes sir. Please do the needful.

Worker: Ok, if you say so, I guess...

17

u/WoodsLovelyDarkNDeep 6h ago

People act like we’ve only had deepfakes since 2022 but that’s not the case.  They’ve been around since at least 2017 and they were so good by 2018 the famous Obama deepfake by Jordan Peele was done

Somehow this narrative got set that so started in 2022 but that’s not true.  Corridor was going crazy things with AI fakes long before then 

Sure complete generated AI video started in 2022 but having a human behind the AI as motion tracking which this could’ve been would’ve been very believable 

7

u/Imwrongyourewrong 6h ago

Yeah and the low quality could be linked to bad internet connection

→ More replies (1)

11

u/ThePokemon_BandaiD 3h ago

In a big zoom call with a lot of people, the resolution on any one person’s video/audio feed is pretty low, so I can see how someone wouldn’t notice.

24

u/iambecomesoil 5h ago

This person was put in charge of too much money and power for the level of critical thinking skills they had!

People tend to believe that people with access or in charge of large amounts of money are somehow smarter than the AVERAGE population.

No. There's smart people and there's plenty of fucking complete dopes. It wasn't meritocracy that got the majority of them there. It was birthright.

7

u/Ana-Luisa-A 5h ago

Not to mention that it was probably low quality call

→ More replies (2)
→ More replies (3)

3

u/lalospv 6h ago

Worker was involved.

→ More replies (1)

4

u/FLBrisby 4h ago

What's that disorder where you genuinely believe everyone you know has been replaced by an imposter? Capgras syndrome?

→ More replies (2)

12

u/JoshDM 7h ago

This person was put in charge of too much money and power for the level of critical thinking skills they had!

Firm was in Hong Kong. Possibly a culture where you don't question orders.

→ More replies (1)

5

u/K_Linkmaster 7h ago

TBC of Tire Kingdom fame requires a voice print to access their 401k. So TBC is recording and keeping that for later use. This problem is pervasive

→ More replies (1)

8

u/LimpConversation642 3h ago

Deep fakes were obvious as hell back then.

they never were. it's just that they become easier to make. Think of dead actors coming to life in Star Wars. It was years before AI and it's still a deep fake, yet they were amazing. It's just that it took a lot of time and money to make those.

But for 25mil I think someone took the time and effort.

→ More replies (2)
→ More replies (43)

123

u/Kriznick 7h ago

So I heard about this and I've always wondered if this was just fraud on the worker's part.

125

u/sick_rock 6h ago edited 6h ago

I work at a bank. Either this case or a similar one was presented as a case study during one of our cybersecurity training.

It is quite believable that the poor worker was duped. Imagine you are minding your business at work and suddenly you get a zoom call from the CFO or CEO telling you that there is a confidential deal and you need to make certain payment immediately. The CEO is staring at you through the zoom and it seems genuine. You want to check the transaction details and maybe with your boss, but your CEO yells at you and threatens that you are gonna cost the firm millions because of how dumb and slow you are and you are gonna get fired. A lot of people would get nervous and make the payment.

A lot of people in the comments are quite confident that they would never fall prey to such tactics, but chances are they are overestimating themselves. A lot of social engineering goes into these elaborate scams, it's not like Nigerian prince mails which are spammed to everyone to hook a few stupid ones.

The company shouldn't have given someone singular authority to transfer such large sums, there should've been maker/checker protocol.

52

u/Gustav55 6h ago

not to mention a bunch of higher ups are also on the call, it seems much more legit when you've got over half a dozen of the major players at your company staring at you.

And prior to this i don't think anyone would even conceive that it was possible to fake the majority of a companies leadership on a single call.

13

u/MeccIt 6h ago

suddenly you get a zoom call from the CFO or CEO telling you that there is a confidential deal and you need to make certain payment immediately.

Ooops, I dropped the zoom call, let me just phone them back directly to get the details.

30

u/MidwesternLikeOpe 5h ago

You underestimate how aggressive scammers can be. I almost got scammed at a job, a vendor said an order wasn't fully paid and we needed to pay them the full amount, they were dropping by for the remaining amount in cash.

They pretended to be my boss but joke was on them bc I had my boss's number and it wasn't him calling. But they were super aggressive about coming by and shutting down our store and firing everyone.

I saw through it though I was a little flustered bc of the aggression. It was a scam running through our corporation and another manager at a different store was reduced to tears by the threats.

I watched a YouTube video of an older lady being scammed by feeding thousands of dollars into a bitcoin machine. The scammer told the cop who intervened to shut up and stand down, don't interrupt the situation. The cop literally had to get into a shouting match with the scammer over the phone.

21

u/msg_me_small_butts 4h ago

I had a case where scammers called pretending to be a vendor we work with. Caller ID showed the call as coming from the vendors usual number but it was spoofed somehow. The scammers were totally friendly and professional and had normal American accents. There was very little to give away that something was wrong.

Ironically the thing that tipped me off is that vendor kind of sucks and having someone on the phone that speaks clearly and is easy to work with was a huge red flag

→ More replies (1)
→ More replies (4)

11

u/nevergnastop 7h ago

He was ai the whole time! It's ais all the way down

194

u/Assholesneighbor 6h ago

I was an operations manager for a larger hotel chain that had several properties. There was a scam where someone would call the night auditor working the front desk at like 3AM and tell them they’re the owner and they need the funds from the safe for an emergency. Night auditors always had access to the safe to balance out the drawers and sales… So, some person getting paid minimum wage takes the few thousand dollars out of the safe and just hands it to a random person who comes inside posing as the owner or owners friend.

This happened like 6 times across several properties before ownership finally was like “what the fuck is going on” and we had a mass meeting. It’s completely the companies fault, but it always amazes me how several different night auditors fell for it and used absolutely no critical thinking… Just going through the motions of work, I guess.

41

u/Hot_Aside_4637 5h ago

Was their name "Mr. Patel"? It's a common scam

32

u/Assholesneighbor 5h ago

No, but very similar! I know, it was a common scam. It was just crazy to me that even after emails were sent and people were notified, it STILL happened.

It was also kind of an attest to our ownership as well… Anytime they came to properties they always came to walk with me and the GM and just nitpick the property and the employees. Obviously, common in the industry, but even when sales were great and the hotel occupancy stayed above 85%, they NEVER came down to give pats on the back…

So, I guess when the intimidating owner you always see yelling at your managers calls you in the middle of the night, you just listen!

11

u/spezial_ed 5h ago

Or they’re in on it? 

24

u/Assholesneighbor 5h ago

No, it was a common scam. They took advantage of the night auditor position having access to funds, usually being a minimum wage employee, and an employee that doesn’t interact with management as often…

It happens all the time still, but many of the bigger franchises put systems in place to stop it. I’m sure boutique hotels still get hit with it.

→ More replies (1)

144

u/funwithdesign 7h ago

This was at least 3 years ago, I would really like to see what these deepfakes looked like.

103

u/TCpls 6h ago

Deepfakes were better 3yrs ago than people are remembering. They just weren’t being done with the same exact tools commercialized today. Not you, but some people seem to not understand the Will Smith spaghetti video is different.

3yrs ago example of a deepfake scam: https://youtu.be/s-boRXkZPQE?si=8bENj0TDIJWJpnWA

Let’s be honest, there are definitely alot of older people who would fall for this sort of thing. I mean it’s 2026 and people still fall for Indian call scammers claiming to be Microsoft and create passwords that include birthdays and pet names 70% of the time.

11

u/SarlacFace 5h ago

Lol Elon has never sounded so lucid in his life. Dude is a stuttering stumbling mess.

This was awful 

15

u/bradygilg 6h ago

That is exactly as bad as I thought it'd be.

10

u/Professional_Elk_489 5h ago

Man this looks super fake

8

u/CreateNewCharacter 4h ago

I knew a woman in 2013 that was telling me about how her pictures had been modified with CGI on the deep web. The tools change, but the actions don't.

→ More replies (3)

41

u/No_Throat_2356 7h ago

CFO & everyone else inexplicably eating spaghetti with their 14 fingers. Just another Tuesday! 

6

u/eightfivethreezerofi 6h ago

What if they specifically targeted companies whose CEOs have 14 fingers?

3

u/tevs__ 6h ago

My man!

6

u/photoggled 5h ago

I mean this is a class of people who just burned $3T on a word guesser machine that guesses the words wrong 10% of the time. It's not a meritocracy out there.

→ More replies (4)
→ More replies (2)

3

u/chefkoch_ 6h ago

Deepfakes from one picture and a few  seconds of voice are good enough to fool half the participants.

3

u/theLuminescentlion 4h ago

Deep fakes where you map someone else onto a real person were still close to believable back then. Especially on a webcam. AI was not generating believable videos then but that doesn't matter to deep fakes.

→ More replies (5)

22

u/OneBillPhil 6h ago

This all just sounds like terrible, basically non-existent internal controls. 

67

u/chefkoch_ 8h ago

2 years old and new AI models seem to convince more than 50% that they are human.

18

u/Thismyrealnameisit 7h ago

That’s carbonism. Silicon based life forms should not be discriminated any more than the Belgians.

19

u/PlagueofSquirrels 7h ago

Please don't use that kind of filthy language in this sub

12

u/Horror-Run5127 7h ago

A B****** waffle sounds good right now

→ More replies (1)
→ More replies (1)

23

u/GeneralCommand4459 6h ago

This is why I never do anything my boss asks me to.

162

u/reepobob 8h ago

Jesus, this scary. We’ll need to have code phrases like Order of the Phoenix members soon.

63

u/Malbethion 8h ago

And a code book, since any time you use the code then the info is out there and can be replicated.

44

u/mrbobcyndaquil 7h ago

One time pads carried physically by courier making a comeback is not what I expected.

8

u/Nazamroth 7h ago

scoffs I only communicate via carrier pigeons, I'm not even affected. I even sent this message to my guy with a computer via one.

3

u/B0Boman 3h ago

Trusting government drones to carry your messages sure is a choice

3

u/PM_ME_MY_REAL_MOM 5h ago

even OTPs are vulnerable at this point, as companies like Palantir, and analogously capable intelligence agencies, are using long-range audio surveillance combined with frequency-pattern analytics to infer offline written and typed content in virtually every room with a window.

it might be possible to avoid this capture by writing the plaintext using keypads whose assigned characters are randomized each keypress, like some bank apps have been using for PINs for a few years now

→ More replies (2)

10

u/dashingThroughSnow12 7h ago

Yeah, my family has a code word for situations like this. Only exchanged in person. Burned upon use by any member in the family.

10

u/Extreme_Mobile_6690 7h ago

For situations like getting into an AI recreation of a work meeting? You must be burning through those words at insane speeds...

6

u/dashingThroughSnow12 6h ago

“Situations like this” being needing to send money to a previously unused destination or other emergency that can’t wait until in-person.

13

u/Blarg0117 7h ago

Albuquerque, New Mexico

→ More replies (1)

7

u/Timelymanner 7h ago

Not entirely necessary, most places have procedures to transfer funds. If this is like most scams, he probably ignored most of them and just directly transferred the funds himself. Or gamble them access to the companies bank information. Scammers are good at tricking people to blindly trust them.

He should have questioned if this is how they normally handle large projects. He should have first told the scammers to submit an invoice, or work order to accounting. Then having accounting verify everything with all banks and financial institutes involved. With that amount of money I’m sure the company would have had to run it past any legal department to make sure things were legal.

If none of these things were a thing, then this company has larger problems if higher ups can just randomly sign away company funds, or share banking info. Seems like a recipe for future internal theft.

4

u/Malphos101 15 7h ago

Or just...require two+ physical in person signatures on any large amount.

Like every other major business has done for all of modern history.

3

u/matt95110 7h ago edited 6h ago

Every family should have a code word to know if they are real or under duress.

3

u/haunter_ 6h ago

Jesus this is so scary!

For me, it seems obvious that this "finance worker" was actually part of the scam and was happy to give away $25 milly and say "omg you guys we totally just got scammed!"

Probably received a generous kickback for 'pretending to fall for a deepfake oopsy haha!' and helped the firm write off some kind of losses or something

3

u/Cpt_Ohu 6h ago

It has been recommended to do this with close family members. If you get a call from someone, and it is about money, legalities or sensitive information, have a code phrase ready to thwart the increasingly sophisticated AI voice generators that by now may be able to fake your child's or spouses voice from a single recording.

→ More replies (1)
→ More replies (17)

10

u/Lastliner 7h ago

How does one guy have so much authority to single handedly transfer funds with no supporting and no checks

33

u/dedwards024 8h ago

Inside job

20

u/StaticSystemShock 4h ago

And all fucking dumbasses want us to plaster our selfies and IDs online for age verification and then wonder when said faces and informations are used for scams. This is one of reasons why I'm so militantly against it. It solves nothing but creates so many problems people are not even aware of it.

→ More replies (2)

17

u/WestTransportation12 8h ago

See I’m too paranoid I would end up calling the guy whose my boss after the meeting to confirm details

8

u/MrCoolGuy42 7h ago

In the near future this is how old people are going to get scammed too. Or at least with phone calls from people that have duplicated voices of their families

4

u/uberfission 6h ago

Considering those scams have been going on for years before AI voice changing became a thing, it's absolutely going to be a golden era for scammers.

→ More replies (3)

32

u/DM_Me_Pics1234403 6h ago

Hiring two people costs more money than hiring one person. In exchange for that cost, you have a more robust system where you can segregate duties and have levels of approval.

Hiring one person costs less than hiring two people. In exchange for those savings, you are create a frail system where a single failure can create a security breach.

When I read something like this, I think to myself that this was part of managements plan. They accepted that hiring less people would increase the chances of this happening. They preferred the savings.

10

u/newdawn-newday 5h ago

Yes, this type of mistake has everything to do with management structure. Especially when you factor in high turnover, experienced workers getting pushed out in favor of over-worked new employees.

7

u/zazzy440 7h ago

What’s to keep someone from deepfaking Putin and using it to browbeat some midlevel Russian officer somewhere to launch a nuke?

6

u/Lord-Benjimus 6h ago

All the procedures, codes/passwords, chain of command stuff. There is a large process to most government things, even banks usually require multiple people to sign off on something before money is sent in these quantities.

5

u/zazzy440 6h ago

You have more faith in the Russian system than I do.

→ More replies (1)
→ More replies (2)

7

u/Valuable-Trust-1757 7h ago

Yup. That’s why banks require you to complete a captcha before joining a Teams call.

5

u/cgsumter 7h ago

In the last couple of weeks both Microsoft and Google have both strongly suggested I upload a video selfie of myself for validation purposes. Seems extermely suspicious to me.

18

u/optre1 1h ago

what if the call was real? and the coleagues used him as an escape goat for fraud?

•

u/IMasterCheeksI 39m ago

Escape Goat

•

u/NachoWindows 28m ago

I like it. Let’s keep it.

→ More replies (1)

•

u/the_king_of_sweden 42m ago

I think you mean scapegoat

•

u/ComprehensiveWar1018 40m ago

Then his colleagues must have taken him for granite

→ More replies (1)

11

u/_Spastic_ 7h ago

Imagine one person being able to move that much cash, without other people and verifications involved. Crazy.

11

u/Thebandroid 7h ago

I'd love to see the recording of that video, $10 says it was just a bunch of blank screens and voice changers, he didn't say anything because he didn't want to turn his camera on either

5

u/Goudinho99 7h ago

I work for a massive financial company and our cybersecurity training now includes this scenario!

9

u/[deleted] 7h ago

[deleted]

9

u/Captain-Cadabra 6h ago

Opens email

Hi it’s me, [your boss]. I’m too busy in a meeting right now, but can you discreetly send me a $25,000,000 iTunes gift card? Don’t respond to my work email, I’m too busy. Just send it to boss@urwerk.biz

→ More replies (1)

5

u/olivebars 6h ago

If that’s how modern business worked, the entire market would seize within a day

→ More replies (3)

3

u/omgFWTbear 6h ago

I had some scammers try a much smaller version of this on me, pretending to be the CEO, and they assumed the CEO talked to me like a cartoon boss, yelling at the peon! which gave away the game immediately.

I followed their instructions meticulously, mailing $500 of Arby’s gift cards to a PO Box in upper Montana. Or so my texts claim. I hope our Very Important Client inks that deal on the basis of … pocket change … the CEO needed urgently… in gift card form …

The Arby’s was my idea though

→ More replies (1)
→ More replies (25)

9

u/kdailyone 6h ago

Work in finance. Can confirm that several people I know could be put in a conf video call with cardboard cutouts and still be tricked.

Don't confuse people working in finance with any sort of bar for basic intelligence or critical thinking.

→ More replies (1)

5

u/CarlJustCarl 7h ago

I’m more interested in how we can make a realistic AI fake to make it look like we are attending a meeting.

4

u/catfroman 6h ago

Plot twist: that guy was on vacation and letting a deepfake do all his work

4

u/Electrical-Drama-755 6h ago

All this tech and we're going to have to go back to pencil and paper b/c we can't trust anything digital.

3

u/FunkMasterPope 7h ago

This finance worker was dumb as rocks. They got phished into a video call and then fell for ai with years ago quality?

3

u/civver3 6h ago

What kind of podunk company has multimillion dollar transfers where the only control is an unsecured videoconference?

3

u/luvz 6h ago

This was 2024 and the video was prerecorded. Had to be an absolute moron in a company with absolutely zero controls.

3

u/vector_o 6h ago

Gotta love it when the workplace is so toxic you aren't allowed to question it if someone acts weird as shit

→ More replies (1)

5

u/COC_410 6h ago edited 5h ago

Sucks that they learned the hard way but this just goes back to putting face to face meetings as the gold standard. Lowkey thought it sucked that everything would be through video calls.

5

u/dfddfsaadaafdssa 4h ago

My boss, who is great friends with the CISO at my company, got got about a month ago because someone cloned the CISO's voice. Thankfully our systems are so robust that nothing of consequence happened but still.

The stuff you hear about is real and it happens everywhere. Even I got SIM swapped a couple of years ago. If I hadn't moved all accounts off of SMS 2FA where possible a few months prior I would have been screwed.

4

u/AnonymousAggregator 1h ago

Is there proof of this meeting taking place, or this persons account?

→ More replies (1)