Cybersecurity News

You probably heard of NFTs (non-fungible tokens). They are receiving a lot of interest over the last several months. I did some digging and realized that there are some bigger issues with the standards and various interpretations and implementations of it, and how centralized many offerings are. What is an NFT The idea behind it is simple, use a blockchain (and cryptography) to assign and be able to proof ownership over a specific piece of digital content.

Next week (on March, 9th 2021) I will be speaking at the Hong Kong Information Security Summit 2021. 歡迎, 你好! I was invited to share my thoughts around protecting the modern (and remote) workplace. Of course, my talk is addressing this topic from a red teaming point of view. Conference details are here. The adversary will come to your house The name of the talk is “Red Team Strategies for Helping Protect the Modern Workplace” which might seem less creative, but there is some (hopefullly) good and interesting information in my talk.

The other day I read this blog post about “The Death of Manual Red Teams” and I thought I’d take a moment to comment on it to provide an alternative perspective. In my opinion the premise of the blog post is backwards, highlighting a lack of understanding of what red teaming is about. For instance the following sentence in the post seems quite incorrect: “Red teaming is the process of using existing, already known security bugs and vulnerabilities to hack a system.”.

The Kindle (ebook) edition of “Cybersecurity Attacks - Red Team Strategies” is currently free on Amazon. Grab your copy while it lasts and spread the word!

The other day Microsoft published a great deep dive around the second stage payloads and hands-on hacking activities of the Solarwinds/Sunburst incidents that where uncovered late 2020. One thing that is so interesting is the use of off the shelve Cobalt Strike tooling and templates for command & control. After doing all the hard work and customization to seamlessly backdoor binaries the adversaries sem to use Cobalt Strike. Intel for Red Teamers Although, they did add customizations with some interesting features, which are interesting for red teamers to be aware of. For instance each instance of the zombie would be unique in name, folder locations, etc. to make it more difficult to identify in environments.

Survivorship bias is an interesting thing that we can observe nearly daily. It is the success stories of exponentially growing startups, the motivational speaker who shares their insights on how to be successful and so forth. What is it exactly? What is survivorship bias? Wikipedia defines it as “…the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.”

Security metrics are an interesting topic. Over the years I used “scores” as a tool to identify and shine light on problematic areas or highlight lack of engineering and security quality of certain teams. A security score should not seen as an objective or absoulte measure, but it allows to compare systems with each other at a relative scale, and by sharing the score it makes people ask questions.

Today FireEye shared that they were victim of a cyberattack and internal red teaming tooling was accessed by adversaries. More details in this NYT article. This reminded me that I wanted to do a post on actively protecting pen testers and pen testing assets for a while. Against persistent adversaries it is only a matter of time when they succeed, not if they will succeed. The big question is do you know when an adversary starts poking around, and when they succeed?

What a journey it has been. I wrote quite a bit about machine learning from a red teaming/security testing perspective this year. It was brought to my attention to provide a conveninent “index page” with all Husky AI and related blog posts. Here it is. Machine Learning Basics and Building Husky AI Getting the hang of machine learning The machine learning pipeline and attacks Husky AI: Building a machine learning system MLOps - Operationalizing the machine learning model Threat Modeling and Strategies Threat modeling a machine learning system Grayhat Red Team Village Video: Building and breaking a machine learning system Assume Bias and Responsible AI Practical Attacks and Defenses Brute forcing images to find incorrect predictions Smart brute forcing Perturbations to misclassify existing images Adversarial Robustness Toolbox Basics Image Scaling Attacks Stealing a model file: Attacker gains read access to the model Backdooring models: Attacker modifies persisted model file Re

In this post we will explore Generative Adversarial Networks (GANs) to create fake husky images. The goal is, of course, to have “Husky AI” misclassify them as real huskies. If you want to learn more about Husky AI visit the Overview post. Generative Adversarial Networks One of the attacks I wanted to investigate for a while was the creation of fake images to trick Husky AI. The best approach seemed by using Generative Adversarial Networks (GANs). It happened that right then deeplearning.ai started offering a GAN course by Sharon Zhou.

There are plenty of examples of artificial intelligence and machine learning systems that made it into the news because of biased predictions and failures. Here are a few examples on AI/ML gone wrong: Amazon had an AI recruiting tool which favored men over women for technical jobs The Microsoft chat bot named “Tay” which turned racist and sexist rather quickly A doctor at the Jupiter Hospital in Florida referred to IBM’s AI system for helping recommend cancer treatments as “a piece of sh*t” Facebook’s AI got someone arrested for incorrectly translating text The list of AI failures goes on…

You might have heard about “NAT Slipstreaming” by Samy Kamkar. It’s an amazing technique that allows punching a hole in your routers firewall by just visiting a website. The attack depends on the router having the Application Layer Gateway enabled. This gateway can be used by anyone inside your network to open a firewall port (totally by design). Protocols such as SIP (Session Initiation Protocol) use it. What I will focus on in this post is the Application Layer Gateway (ALG) and SIP. In particular to understand how these can be abused by an adversary.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: The attacks I want to investigate, learn about, and try out In this post we are going to look at the “Repudiation Threat”, which is one of the threats often overlooked when performing threat modeling, and maybe something you would not even expect in a series about machine learning.

My GrayHat Red Team Village talk “Learning by doing: Building and breaking a machine learning system” is now live on YouTube. Check it out: https://www.youtube.com/watch?v=-SV80sIBhqY and smash the Like button! :D Question? I thought of turning the content into a hands-on workshop. Let me know if that would be something that would you would attend? Trying to see if there is interest. Cheers, Johann Twitter: @wunderwuzzi23

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: Some of the attacks I want to investigate, learn about, and try out A few weeks ago while preparing demos for my GrayHat 2020 - Red Team Village presentation I ran across “Image Scaling Attacks” in Adversarial Preprocessing: Understanding and Preventing Image-Scaling Attacks in Machine Learning by Erwin Quiring, et al.

A few years back the Blue Team of a company asked to be targeted in a Red Team Operation. That was a really fun, because Rules of Engagement commonly prevent targeting Blue Teams. Blue’s infrastructure, systems and team members are often out of scope, unfortunately. Blue team infrastructure is a gold mine for credentials, recon but also for remote code execution! Often companies do not have adequate protection, procedures (MFA, multi-person attestation), monitoring and auditing in place when it comes to accessing data from endpoint agents. There is also frequently a lack of knowledge on what Endpoint Agents are capable of doing.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: Some of the attacks I want to investigate, learn about, and try out I wanted to explore the “Adversarial Robustness Toolbox” (ART) for a while to understand how it can be used to create adversarial examples for Husky AI.

For GrayHat 2020 I was asked to create a short intro video for my Red Team Village talk “Learning by doing: Building and breaking a machine learning system”. So I put my green screen to good use and recorded this short clip for Red Team Village. Here is the link to the clip on Twitter: Hope you like it. :) The talk will be October, 31st 2020.

There is a lot of discussion around terms such as red team, attack team, pentest, adversarial engineering or offensive security team and similar ones. I typically stay away from the (sometimes passionate) discussions that ensue whenever this topic comes up. Personally, I think a good strategy is to define programs and teams who operate in this space by what services the team (or teams) provide(s) to the organization. The business groups, blue team, developers, engineers, employees and clients are the customers.

While building “Husky AI” I started working a lot with Microsoft’s VS Code Python extension. It is a super convinient way to edit Jupyter Notebooks. I just use VS Code’s Remote SSH feature to get to my Linux host and work on modeling and testing there. When threat modeling “Husky AI” I identified backdooring of third party libraries and development tools as a potential issue to be aware of.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: The attacks I want to investigate, learn about, and try out We talked about creating adversarial examples and “backdoor images” for Husky AI before. One thing that we noticed was that an adversary with model access can very efficiently come up with adversarial examples.

Excited to announce that I will be presenting at Grayhat - Red Team Village on October 31st 2020. The presentation is about my machine learning journey and how to build and break a machine learning system. If you follow my blog, you can guess that there will be lots of discussion around “Husky AI”. The bits and pieces that make up a machine learning pipeline, and how to threat model such a system.

This was also presented at BSides Singapore 2020. The slides are here and YouTube link is here. The origins of the Shadowbunny A few years ago, around 2016, I went on a relaxing two weeklong vacation. It was great to disconnect from work. I traveled to Austria, enjoying hiking in the mountains, and exploring Vienna. When I came back to the office, the team had placed a giant bunny teddy into my chair. In retrospect, it seemed a legitimate replacement for the manager, as hardly anyone seemed to have noticed my absence.

This year one of my goals was to learn about machine learning and artificial intelligence. I wrote about my journey before - including what classes I took and books I read, the models and systems I built and operationalized, threat modeling it to learn about practical attacks and defenses. My goal is to be knowledge enough in the AI/ML space enough to be able to help bridge the gap between research and operational red teaming - by doing practical things with life systems.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: The attacks I want to investigate, learn about, and try out Mitigations: Ways to prevent and detect the backdooring threat During threat modeling we identified that an adversary might tamper with model files. From a technical point of view this means an adversary gained access to the model file used in production and is able overwrite it.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. Overview: How Husky AI was built, threat modeled and operationalized Attacks: The attacks I want to investigate, learn about, and try out The previous post covered some neat smart fuzzing techniques to improve generation of fake husky images. The goal of this post is to take an existing image of the plush bunny below, modify it and have the model identify it as a husky.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. There are the two main sections of the series - more content will be added over time: Overview: How Husky AI was built, threat modeled and operationalized Attacks: The attacks I want to investigate, learn about, and try out The previous post covered basic tests to trick the image recognition model. This included generating “bad” images with solid colors or entire random pixels.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see related posts. The previous four posts explained the architecture and how Husky AI was built, threat modeled and deployed. Now it’s time to start the attacks and build mitigations. The appendix in this post shows all the attacks I want to research and perform in this series over the next few weeks/months.

This post is part of a series about machine learning and artificial intelligence. Click on the blog tag “huskyai” to see all the posts, or visit the machine learning attack series overview section. In the previous post we walked through the steps required to gather training data, build and test a model to build “Husky AI”. This post is all about threat modeling the system to identify scenarios for attacks which we will perform in the upcoming posts.

This post is part of a series about machine learning and artificial intelligence. In the previous post we walked through the steps required to gather training data, build and test a model. In this post we dive into “Operationalizing” the model. The scenario is the creation of Husky AI and my experiences and learnings from that. Part 3 - Operationalizing the Husky AI model This actually took much longer than planned. Since I used TensorFlow, I naively thought it would be very straight forward to implement a Golang web server to host the model. Turns out that TensorFlow/Keras is not that as straightforward to integrate with Golang, it requires a lot of extra steps. So, I ended up picking Python for the web server.