In October, VMware fixed a critical remote code execution vulnerability in its vCenter Server (CVE-2023-34048) and Cloud Foundation enterprise products that are used to manage virtual machines across hybrid clouds. It has now come to light that a Chinese cyberespionage group had been exploiting the vulnerability for 1.5 years before the patch became available.
“These findings stem from Mandiantâs continued research of the novel attack paths used by UNC3886, which historically focuses on technologies that are unable to have EDR deployed to them,” researchers from security firm Mandiant said in a report late last week. “UNC3886 has a track record of utilizing zero-day vulnerabilities to complete their mission without being detected, and this latest example further demonstrates their capabilities.”
Suspicious VMware log entries date back to 2021
In June 2023, Mandiant documented how the Chinese group it tracks as UNC3886 exploited a zero-day authentication bypass vulnerability in VMware Tools (CVE-2023-20867) to deploy backdoors inside guest VMs from compromised ESXi hosts. That attack flow described by Mandiant started with hackers first gaining access to vCenter servers and then using known techniques to extract cleartext credentials for the vpxuser account for all ESXi hosts attached to the server. This allowed them to access those hosts and exploit CVE-2023-20867 to deploy malware.
However, the password for vpxuser — an account created on ESXi hosts automatically when associated with a vCenter server — is encrypted by default. On a fully patched vCenter system, cracking the passwords requires root access. So, how did attackers gain root access to vCenter servers in the first place? By exploiting the CVE-2023-34048 vulnerability that was later patched in October 2023.
Mandiant’s forensic analysts found a commonality on compromised vCenter systems where the crash logs located in /var/log/vMonCoredumper.log showed the “vmdird” service crashing minutes prior to attackers deploying their malware. After sharing this observation with VMware’s product security team along with memory core dumps of the crashed vmdird process, the conclusion was reached that the crashes are closely aligned with the behavior observed during CVE-2023-34048 exploitation.
The CVE-2023-34048 flaw is an out-of-bounds write in the implementation of the DCERPC protocol that leads to a crash and arbitrary code execution. The flaw can be exploited remotely over the network.
“VMware strongly recommends strict network perimeter access control to all management components and interfaces in vSphere and related components, such as storage and network components, as part of an overall effective security posture,” VMware said in a FAQ document associated with the vulnerability. “The specific network ports involved in this vulnerability are 2012/tcp, 2014/tcp, and 2020/tcp.”
Mandiant noted that it has observed signs of these crashes in logs on compromised environments going back to late 2021 and early 2022, but the vmdird core dumps were not present on those systems. A memory core dump is generated automatically when a process crashes and VMware’s default configuration is to keep these core dumps on the system for an indefinite amount of time. The fact that they were removed on many systems suggests the attackers purposely deleted them to cover their tracks.
While publicly reported and patched in October 2023, Mandiant has observed these crashes across multiple UNC3886 cases between late 2021 and early 2022, leaving a window of roughly a year and a half that this attacker had access to this vulnerability. Most environments where these crashes were observed had log entries preserved, but the vmdird core dumps themselves were removed. VMwareâs default configurations keep core dumps for an indefinite amount of time on the system, suggesting the core dumps were purposely removed by the attacker to cover their tracks.
Organizations should already have the patches for CVE-2023-34048 applied. However, the revelation that hackers exploited this flaw for 1.5 years as a zero-day is concerning and should prompt further investigations into environments, especially for the UNC3886 indicators of compromise and backdoors documented by Mandiant.