One way Optiv helps its clients is by speeding up their vulnerability management process. In a recent project we sought to reduce mean time to remediate (MTTR) by accelerating the processes from vulnerability discovery to the deployment of a corrective system update.

The vulnerability management process for user workstation and on-premise servers is ripe for revision. Taking cues from current systems management and application development processes, legacy vulnerability management can be transformed and accelerated by adopting two components:

The technologies required to incorporate these components into the vulnerability management process are robotic process automation (RPA) and systems management tools that offer the orchestration of vulnerability scan data, patch correlation and rules-based deployment through patch management solutions.

During our research we found that there’s no standardized method to model and analyze vulnerability management workflows. To have a better method to model these workflows Optiv created a “vulnerability management pipeline” concept. The pipeline is based on a set of building blocks that align with the major and minor phases organizations perform during the vulnerability management process.

In order to improve remediation times, an organization will first need to understand what factors are contributing to the overall time to remediate. Rather than building on a dated model, the goal of the pipeline is to break the process into better defined and measurable phases. This post will focus on how Optiv achieved automated end-to-end discovery to remediation for Windows workstations. An example of this pipeline is shown below.

Image

Figure 1 – Windows workstation remediation pipeline

Lab Architecture

Image

Lab Environment

Integrating Vulcan Cyber Into the Remediation Pipeline

Image

Figure 2 – Adding Vulcan Cyber to a basic vulnerability management configuration

Vulcan’s solution was added to the remediation pipeline as a method to correlate Tenable.io vulnerability scan data with the appropriate corrective action. The initial design is shown above. Using the API integrations between Tenable.io, Vulcan Cyber and Microsoft SCCM, the flow of the pipeline is as follows:

At this stage an operator can initiate patch deployment from SCCM to the device collection.

An example Vulcan runbook used in this proof of concept is shown in Figure 3 below. When Vulcan receives vulnerabilities with fixes from Tenable.io matching asset names that begin with PRS-KS, it creates an SCCM remediation action.

Image

Image

Figure 3 – Example Vulcan runbook

Deployment Automation

In order to automate the “last mile” of the Windows remediation pipeline Optiv used the automation functionality provided in Microsoft’s Power Automate and Azure Automation. In figure 4 the additional components of the remediation pipeline are highlighted in green.

Image

Figure 4 – Additions to the vulnerability management configuration

An action is added to the Vulcan lightweight automation runbook that sends an email with all of the aggregated information to a service email account. The Playbook note, which is optional, was added as a way for Power Automate to down select only certain emails as a trigger. This will be explained further in the steps below.

Image

Figure 5a – Kicking off a Vulcan runbook

A Power Automate flow was created to assist with the automation. The flow is triggered by an email from hello@vulcan[.]io that has attachments.

Image

Figure 5b – Power Automate flow trigger from Vulcan runbook

The next step in the flow is to narrow the scope of the emails with attachments by performing a regular expression search for a particular note in the body of the email. This information was added in the Vulcan action shown above in figure 5b.

Image

Figure 6 – Conditional step

If the condition is met when the body of the email matches the regular expression the flow continues on to the next step, which is an Azure Automation runbook.

Image

Figure 7 – Azure Automation runbook

The details of this Azure Automation runbook are shown in the PowerShell code below. When the code is executed on the hybrid-worker, it will perform the “last mile” tasks that were still manual in the first iteration of the remediation pipeline.

This particular runbook is rather static, as it lists a specific software update group, device collection and deployment name, but it can be altered to include others. The script is shown as the first action step highlighted in red in the flow below.

Image

Figure 8 – Power Automate Cloud Flow

Patch Unit Testing with Robotic Process Automation

Optiv recently surveyed a sample set of clients regarding vulnerability remediation. All of the clients surveyed responded they would allow for end-to-end automated remediation, with no human intervention, if user testing was completed. Optiv has shown that user acceptance testing, post update, can be automated with the use of robotic process automation. Not only can the testing be automated, but it can be an integrated step of the pipeline.

While this post is not intended to provide detailed guidance on the use of RPA, it is important that once user workflows are captured, they are tested repeatably to ensure a consistent and expected outcome. Once confident in the RPA’s expected outcome of the workflow(s), the execution of the workflows can be added as a step in the pipeline.

Adding unit testing:

Image

Figure 9 – Adding unit testing to the vulnerability management configuration

In the final iteration of the pipeline, Optiv used several Azure Automation runbooks and a Power Automate Desktop flow as additional actions that are added on to the previous Power Automate cloud flow. In order for RPA to be used as a workflow test, Optiv needed to ensure the software updates were applied to the test host. Optiv created an Azure Automation runbook to force the test client to check for updates:

This is what this action looks like in the Power Automate flow.

Image

Figure 10 – Power Automate Flow

After the host is “forced” to run the update a delay timer is executed. The delay timer is intended to provide the host enough time to install the updated. After the time has expired an Azure Automation runbook is used to query that the update was installed.

Image

Figure 11 – Power Automate Flow details

The third Azure Automation runbook that was used is listed below. The runbook checks if the deployment to the host was successful by running the Get-CMDeployment command and checking the NumberSuccess object.

If the deployment was successful the host will execute the user workflow test(s) using Microsoft’s Power Automate Desktop flow.

Image

Figure 12 – Power Automate Desktop Flow

Using a Power Automate Desktop flow, a simple user workflow is executed after the host is patched to ensure that the software or security update does not disrupt any workflow or business process. In Optiv’s proof of concept a single user workflow was tested in the remediation pipeline, but multiple local and web application workflows can be added to this flow to ensure all possible user tasks are tested before scheduling an update for the entire organization.

Automated Remediation Pipeline

With the assistance of Microsoft Desktop flow, user workflow testing is complete. The last step in the automated remediation pipeline is to deploy the software update to a larger device collection in SCCM. Optiv was able to do this using an additional Azure Automation runbook. The runbook below assigns the software update group to an existing device collection that contains additional Windows 10 hosts. These hosts are running the same version of Windows 10 and have the same applications installed as the one tested in the RPA action of the Power Automate flow.

The code above is in the action highlighted in red in the flow below.

Image

Figure 13 – Power Automate Desktop Flow details

Complete Flow:

Image

Figure 14 – Complete flow

This pipeline shows that workstation vulnerability management can be fully automated, including user workflow testing, with no human interaction. In a previous project, Optiv validated that endpoint remediation can be tested using a similar flow with Mandiant Security Validation. This research was completed in 2020 and Optiv was able to create an Azure host, install the MSV agent and perform testing with no human interaction.

In my next Source Zero post I will add the ability to provide efficacy testing, through Mandiant Security Validation, to the existing remediation pipeline, further automating validation of the remediation action.