Cybersecurity News

Elastic Security Labs exposes an attempt by the DPRK to infect blockchain engineers with novel macOS malware.

Elastic Security Labs reveals details of a new campaign leveraging defense evasion capabilities to infect victims with malicious MSIX executables.

Large Language Model (LLM) applications and chatbots are quite commonly vulnerable to data exfiltration. In particular data exfiltration via Image Markdown Injection is frequent. This post describes how Google Cloud’s Vertex AI - Generative AI Studio had this vulnerability that I responsibly disclosed and Google fixed. A big shout out to the Google Security team upfront, it took 22 minutes from report submission to receiving a confirmation from Google that this is a security issue that will be fixed.

This article highlights the essential contributions to the Global Threat Report from the Security Intelligence team, and describes three major phenomena impacting the threat landscape.

BLOODALCHEMY is a new, actively developed, backdoor that leverages a benign binary as an injection vehicle, and is a part of the REF5961 intrusion set.

In this publication, we will walk through this client application’s functionality and how to get started with the tool.

The REF5961 intrusion set discloses three new malware families targeting ASEAN members. The threat actor leveraging this intrusion set continues to develop and mature their capabilities.

Large Language Model (LLM) applications and chatbots are quite commonly vulnerable to data exfiltration. In particular data exfiltration via Image Markdown Injection is quite frequent. Microsoft fixed such a vulnerability in Bing Chat, Anthropic fixed it in Claude, and ChatGPT has a known vulnerability as Open AI “won’t fix” the issue. This post describes a variant in the Azure AI Playground and how Microsoft fixed it. From Untrusted Data to Data Exfiltration When untrusted data makes it into the LLM prompt context it can instruct the model to inject an image markdown element. Clients frequently render this using an HTML img tag and if untrusted data is involved the attacker can control the src attribute.

Learn more about how Elastic Security Labs has been focused on accelerating our detection engineering workflows by tapping into more generative AI capabilities.

During an Indirect Prompt Injection Attack an adversary can exfiltrate chat data from a user by instructing ChatGPT to render images and append information to the URL (Image Markdown Injection), or by tricking a user to click a hyperlink. Sending large amounts of data to a third party server via URLs might seem inconvenient or limiting… Let’s say we want something more, aehm, powerful, elegant and exciting. ChatGPT Plugins and Exfiltration Limitations Plugins are an extension mechanism with little security oversight or enforced review process.

In our previous article, we explored using the GPT-4 Large Language Model (LLM) to condense Linux user sessions. In the context of the same experiment, we dedicated some time to examine sessions that shared similarities. These similar sessions can subsequently aid the analysts in identifying related suspicious activities.

Last month I had the opportunity to attend HITCON in Taiwan for the first time. It’s an annual event hosted by the Hackers in Taiwan organization and CMT stands for the community version. There is a second event for enterprises later this year also - think of it like Blackhat vs Defcon in a way. Conference, Location and Registration HITCON CMT 2023 was a two day event hosted in the east side of Taipei at Academia Sinica.

What happens if an attacker calls an LLM tool or plugin recursively during an Indirect Prompt Injection? Could this be an issue and drive up costs, or DoS a system? I tried it with ChatGPT, and it indeed works and the Chatbot enters a loop! 😊 However, for ChatGPT users this isn’t really a threat, because: It’s subscription based, so OpenAI would pay the bill. There seems to be a call limit of 10 times in a single conversation turn (I tried a few times). Lastly, one can click “Stop Generating” if the loop keeps ongoing. BUT

In this research publication, we'll learn about upcoming improvements to the Windows Code Integrity subsystem that will make it harder for malware to tamper with Anti-Malware processes and other important security features.

In this article, we'll show you how we contextualize rules and events, and how you can leverage call stacks to better understand any alerts you encounter in your environment.

In this publication, we will talk about lessons learned and key takeaways from our experiments using GPT-4 to summarize user sessions.

This video highlights the various data exfiltration vulnerabilities I discovered and responsibly disclosed to Microsoft, Anthropic, ChatGPT and Plugin Developers. It also briefly discusses mitigations various vendors put in place (and triage decisions).   Thanks to MSRC, Anthropic and Zapier for addressing vulnerabilities to help protect their users.

Bring Your Own Vulnerable Driver (BYOVD) is an increasingly popular attacker technique whereby a threat actor brings a known-vulnerable signed driver alongside their malware, loads it into the kernel, then exploits it to perform some action within the kernel that they would not otherwise be able to do. Employed by advanced threat actors for over a decade, BYOVD is becoming increasingly common in ransomware and commodity malware.

Elastic Security Labs dives deep into the recent evolution of the BLISTER loader malware family.

A common attack vector that LLM apps face is data exfiltration, in particular data exfiltration via Image Markdown Injection is a common vulnerability. Microsoft fixed the vulnerability in Bing Chat, ChatGPT is still vulnerable as Open AI “won’t fixed” the issue, and Anthropic just mitigated this vulnerability in Claude. This post documents the Anthropic Claude data exfiltration vulnerability and the mitigation put in place. The Vulnerability - Image Markdown Injection As a quick recap, imagine a large language model (LLM) returns the following text:

This research reveals insights into some of the large-scale malware analysis performed by Elastic Security Labs, and complements research related to the Detonate framework.

ChatGPT is vulnerable to data exfiltration via image markdown injections. This. is. pretty well known. As more features are added to ChatGPT the exfiltration angle becomes more likely to be abused. Recently OpenAI added Custom Instructions, which allow to have ChatGPT always automatically append instructions to every message exchange. An adversary can abuse this feature to install a data exfiltration backdoor that depends on, and only works because of the image markdown injection vulnerability. The TTP is a similar to other post exploitation techniques adversaries are using, like enabling email forwarding rules.

A prompt injection scenario that I, and others, have been wondering about in the past, is the potential risk associated with chatbots being able to analyze images. Could this ability open up the way for Indirect Prompt Injection attacks? Recently, Google added the ability to uploading and analyze images with Bard. And it turns out that it is indeed possible to add instructions to an image, and have the Bard follow those instructions.

Watch out! We’ve recently discovered a variant of RUSTBUCKET. Read this article to understand the new capabilities we’ve observed, as well as how to identify it in your own network.

Google Docs is a popular word processing tool that is used by millions of people around the world. Recently Google added new AI features to Docs (and a couple of other products), such as the ability to generate summaries, and write different kinds of creative content. Check out Google Labs for more info. These features can be very helpful, but they also introduce new security risks. At the moment there are not too many degress of freedom an adversary has, but operating your AI on untrusted data can have unwanted consequences:

In the previous post we discussed the risks of OAuth enabled plugins being commonly vulnerable to Cross Plugin Request Forgery and how OpenAI is seemingly not enforcing new plugin store policies. As an example we explored how the “Chat with Code” plugin is vulnerable. Recently, a post on Reddit titled “This is scary! Posting stuff by itself” shows how a conversation with ChatGPT, out of the blue (and what appears to be by accident) created a Github Issue! In the comments it is highlighted that the Link Reader and Chat With Code plugins were enabled when ChatGPT created this Github Issue here.

Elastic Security Labs observes that the threat behind SIESTAGRAPH has shifted priorities from data theft to persistent access, deploying new malware like NAPLISTENER to evade detection.

Explore JOKERSPY, a recently discovered campaign that targets financial institutions with Python backdoors. This article covers reconnaissance, attack patterns, and methods of identifying JOKERSPY in your network.

OpenAI continues to add plugins with security vulnerabilities to their store. In particular powerful plugins that can impersonate a user are not getting the required security scrutiny, or a general mitigation at the platform level. As a brief reminder, one of the challenges Large Language Model (LLM) User-Agents, like ChatGPT, and plugins face is the Confused Deputy Problem / Plugin Request Forgery Attacks, which means that during a Prompt Injection attack an adversary can issue commands to plugins to cause harm.

This post describes how I found a Prompt Injection attack angle in Bing Chat that allowed malicious text on a webpage (like a user comment or an advertisement) to exfiltrate data. The Vulnerability - Image Markdown Injection When Bing Chat returns text it can return markdown elements, which the client will render as HTML. This includes the feature to include images. Imagine the LLM returns the following text:  This will be rendered as an HTML image tag with a src attribute pointing to the attacker server.