is your security ecosystem inaccurately portraying your ...a digital defense, inc. white paper is...

A Digital Defense, Inc. White Paper

Is Your Security Ecosystem Inaccurately Portraying Your Information Security Risk?CONSEQUENCES OF INTEGRATING WITH A LIMITED VULNERABILITY MANAGEMENT SYSTEM

By Gordon MacKay EVP, Chief Technology Officer

White Paper: Is Your Security Ecosystem Inaccurately Portraying Your Information Security Risk? © 2016 2

BREACHES ARE HAPPENING AT ALARMING RATES. NO ORGANIZATION TOO LARGE OR TOO SMALL IS SAFE FROM THE LIKELIHOOD THAT A CYBER CRIMINAL COULD POTENTIALLY LEVERAGE NETWORK VULNERABILITIES TO GAIN ACCESS TO SENSITIVE DATA.

Frustrated security experts are learning that the success of any information security program is dependent on the accuracy of security intelligence. For the past 18 months, Digital Defense, Inc. has warned the infosec community about a serious limitation found in many vulnerability management (VM) systems that has companies exhausting resources or making far-reaching and often, expensive security decisions based on inaccurate data.

We analyzed the trends and published the study, Vulnerability Management Solutions are Flawed, Leaving your Enterprise at High Risk, warning consumers not to have a false sense of security and misplaced faith in highly touted scanning tools that provide misleading information.

Today the risks have increased. Now organizations are encouraged to create a healthy security ecosystem and integrate with a many solutions in an effort to take a holistic approach to security. However, if organizations are integrating with VM providers that are not producing accurate vulnerability scanning results, critical decisions could be based on faulty data.


INTRODUCTIONWith multiple benefits promised from integrating security information from traditional siloed environments, many organizations are moving towards an integrated and evolving security ecosystem. Bringing together information from various security technologies and solutions along with externally available security threat intelligence, provides an ability to control, measure and reduce security risk. A crucial component of this intelligence, an organization’s network assets and their associated security weaknesses, comes from a Vulnerability Management (VM) solution.

Because important security decisions are dependent upon it, it is highly important that the information stored within and gleaned from VM systems be as accurate as possible. However, most VM solutions that are central to every Enterprise information security program, and essential in gauging network security information risk, contain a serious “hidden” limitation. This software simplified algorithm that is beginning to come to light is interwoven within pattern matching-like algorithms located deep within the foundational core of the most prevalent and widely used automated VM system products and solutions on the market today. The nature of this limitation causes the resulting gap error to become compounded over time, further undermining the usefulness of the data they provide. As a direct consequence of this limitation, correlation of the VM data along with data from other security systems within an organization’s security ecosystem, results in highly inaccurate conclusions as well as higher levels of security risk exposure.

We believe it is important to provide detail around the technical challenge which lies at the root of the limitation, and which has been introduced by even the largest VM vendors in the space. The nature of this challenge, and the reason a solution to this crucial issue is required, is described. Further, the consequences of integrating with a limited VM solution and the harmful ripple effect the faulty VM data has within an organization’s security ecosystem is illustrated.


THE RISE OF THE INTEGRATED INFORMATION SECURITY ECOSYSTEMInformation security solutions have traditionally operated within their own isolated technology island, providing value related to a specific designed solution, but not taking advantage of available information from the organization’s other security solutions. However, within the last five years, many organizations have realized the benefits of bringing together information from these traditionally siloed security solutions, and are evolving their integrated security ecosystem.

There is no ‘one size fits all’ information security ecosystem nor does any vendor offer organizations a fully integrated solution which solves all information security use cases. However, many vendor security solutions have emerged with key integrations with other security vendors that solve some high runner security use cases. Further, most vendors also offer Application Programming Inter-

faces (APIs) allowing their solution to be integrated with complimentary security solutions. In most cases, an organization must evolve their security ecosystem based upon their

security risk and threat models. This requires a particular level of internal IT integration work specific to the organization, as well as ongoing IT operation maintenance.


The following figure illustrates an abstract example of an integrated information security ecosys-tem, which includes networked computer assets, as well as various security solutions such as Intru-sion Prevention Systems (IPS), Identity and Access Management Systems (IAM), Firewalls, Data Loss Prevention (DLP), Vulnerability Management (VM), Governance Risk and Compliance (GRC), Incident Response (IR), Security Information and Event Management (SIEM) systems, all integrated together.

There are many possible security use cases which require integration between one or more secu-rity solutions found within a security ecosystem, but many of these scenarios involve knowledge on security weaknesses. To a large extent, this information comes from a VM system. With this in mind, an example security use case which requires a VM system is introduced, and a key challenge which impacts this use case and many other use cases is shared: network churn over time.

FIGURE 1: Integrated Security Ecosystem

IPS

VULNERABILITYMANAGEMENT

INCIDENT RESPONSE

IAM

SIEM

IAM

GRC

DLP


USE CASE: MARRYING THREAT WITH WEAKNESS INFORMATIONThe following figure illustrates a powerful security use case you may find in many existing security ecosystems. Information from an organization’s IDS system is pulled into a SIEM along with infor-mation from their VM solution. The goal of the integration is to identify instances when the IDS de-tects an imminent or possible attack against a network endpoint (event information from the IDS), and where the given endpoint is vulnerable to the given attack (information from the VM solution).

The figure depicts an example where a possible SQL injection attack is directed towards the endpoint which has IP address 192.168.1.5. The IDS packet is sent to the SIEM which has already received information via the VM system related to this endpoint, as well as related to all other endpoints in the organization’s network. This particular scenario assumes the VM declared no vulnerability findings for this asset. As a result, the SIEM does not alert on the attack and all is well. Or is it?

FIGURE 2: IDS Integrated with VM and SIEM


IDS

SIEM

DATA PACKETSource IP=...Destination IP=192.168.1.5Payload=...SQL Attack...

EVENTIP=192.168.1.5SQL-I Attack

VULNERABILITY FINDINGS

CORRELATE IDS ATTACKWITH

VULNERABILITY INFORMATION


VULNERABILITY MANAGEMENT SYSTEMS ARE CHALLENGEDWhen information is married from two or more different information security data sources, often the integrated solution must correlate events and information that occur at different points in time.

It is often the case that when the dimension of time is considered within the human model of reality, the resulting complexity of our model increases by magnitudes. This is also true for integrations involving two or more information security solutions and especially true when one of these is a VM solution, since they manage assessment information for assessments which occur over time. At the root of this issue is that as time passes, the underlying network that these solutions measure and protect, changes.

Although the correlation rules relating the two data sets

seem quite simple, integrations involving most VM solutions available for purchase, even those of the largest vendors, yield significant errors which may result in dire consequences such as a false negative occurrence of an imminent attack where the system is vulnerable. This is because most of the VM solutions on the market today are challenged in dealing with changes in the underlying network over time. Although the challenge is explained in greater detail in a related whitepaper, Vulnerability Management Solutions are Flawed, Leaving your Enterprise at High Risk, a high level explanation is given on the next page.


Figure 3 illustrates this challenge, reflecting two different and independent vulnerability assess-ments performed at different points in time. The figure illustrates the hosts discovered within each assessment, some of the discovered host characteristics, and the assessed hosts’ association to their corresponding true real world assets.

In this example, Assets A and B have been re-deployed or re-configured since week 1, and Asset C has either been removed from the network (either temporarily or permanently) or was powered down/offline at the time of the second test.

FIGURE 3: Matching Assessed Hosts to their Correct Real World Assets

TIME

SCANWEEK 1

SCANWEEK 2

Vulnerability Assessments Across Time

ASSET A ASSET B ASSET C

Real World Network Assets

IP=3DNS HN=PapayaNETBIOS HN=WhiteMAC=None

IP=192.168.1.20DNS HN=NoneNETBIOS HN=BlueMAC=00-0C-29-AD-9B-44

IP=192.168.1.5DNS HN=CherryBlossomNETBIOS HN=NoneMAC=None

IP=192.168.1.20DNS HN=CherryBlossom

NETBIOS HN=NoneMAC=None

IP=192.168.1.5DNS HN=None

NETBIOS HN=NoneMAC=00-0C-29-AD-9B-44


An ideal VM solution would recognize that changes have occurred since the first assessment, and would match the assessed hosts from the second week’s assessment to the correct counterparts as assessed within the first week’s assessment. However, most VM solutions are limited in dealing with network changes that occur over time and as a result will mismatch assessed hosts to incorrect assets.

The previously referenced whitepaper, which includes a study on the prevalence of change to an organization’s network asset characteristics, such as an asset’s IP Address, Hostname, MAC Addresses and others, reveals that these characteristics change far more often than was once assumed. The findings from this study were quite voluminous; therefore, only a cross section of the results are included in this paper, specifically findings related to server type machines and client type machines. The table below summarizes the network change rate findings for devices deemed Server machines:

The study revealed that, over a three-month period, an average of 4% of the real world Server machines had a change in IP Address (relatively low, but significant in large networks),but 46% of these machines had a change in DNS Hostname and 34% had a change in NETBIOS Hostname.

A summary of the findings for devices which were deemed client machines appears in the table below:

Again, the figures shown indicate the percentage of assets deemed client machines which experienced a change in the given network host characteristic over the course of a three-month timeframe. We see client machines have a much higher network change incidence of the IP

TABLE 1: Server Network Host Characteristic Change Over Time

HOST CHARACTERISTIC % CHANGE

IP Address 4%

DNS Hostname 46%

NETBIOS Hostname 34%

TABLE 2: Client Host Characteristic Change Over Time

HOST CHARACTERISTIC % CHANGE

IP Address 36%

DNS Hostname 42%

NETBIOS Hostname 20%


Address characteristic than that seen for servers. This intuitively makes sense given that client machines, which are used by humans to perform a job function, are often mobile in nature, and as such organizations often configure these to obtain their IP Address dynamically by way of DHCP. The percentage of change in DNS and NETBIOS Hostnames runs high, as did the statistics for servers.

If a VM solution relies on these characteristics to match scanned results to real world assets and/or match scan results to previous assessments, the above results are quite alarming and clearly have serious implications. Assume for the mo-ment that the large sample of networks and devices included in our study is repre-sentative of all networks (which is a fair assumption given the large sample size we used). One observation is that a VM system which makes the assumption that IP Address does not change over time for servers (and therefore uses IP Address to match assessed hosts to their real world asset counterpart), will not correctly match the assessed host to its real world asset 4% of the time across a period of 3 months. One may argue that 4% is a low number, but to put this into perspective, an Enterprise organization having 100,000 Servers and using a VM system which matches in this fashion would have a resulting 4,000 of these servers with out-of-date and/or entirely inaccurate findings after 3 months. As time pro-gresses, more hosts that were not originally mismatched within the first 3 months of assessments could later also experience mismatches. Additionally, the problem compounds itself as time pro-gresses because accurate matching requires an accurate basis; because of past mismatches, the basis is incorrect. This issue is even worse for cases where matching makes sole use of the other characteristics mentioned, since the prevalence of change is even higher than for the IP Address.

Given the much higher churn rate seen among client machines, scanning results comparisons can become virtually useless in a very short timeframe.

An Enterprise organization having 100,000 Servers could have

4,000 out-of-date servers and/or entirely inaccurate findings after

three (3) months.


INTEGRATING WITH VM – A MATCH NOT MADE IN HEAVEN

Let’s revisit our previous use case within the context of the aforementioned VM limitation. Figure 1 illustrates an attack packet passing through the IDS and destined for the internal IP address 192.168.1.5. This attack information is sent from the IDS, to the SIEM. The SIEM also receives vulner-ability information from the VM system, either periodically or on demand (depending upon the inte-gration) for the given asset located at the given IP 192.168.1.5, as well as for all other endpoint assets. The SIEM then draws a conclusion as to whether the asset has vulnerabilities specific to the attack.

Additionally, assume the VM solution has experienced a mismatch error during its matching process within a second week’s assessments because of network change and the issues previously described. The mismatch situation is illustrated in the following figure.

FIGURE 4: Mismatch case of Assessed Host Mismatch to incorrect Asset

TIME

SCANWEEK 1

SCANWEEK 2

Vulnerability Assessments Across Time

ASSET A ASSET B ASSET C

Real World Network Assets

IP=3DNS HN=PapayaNETBIOS HN=WhiteMAC=None

IP=192.168.1.20DNS HN=NoneNETBIOS HN=BlueMAC=00-0C-29-AD-9B-44

IP=192.168.1.5DNS HN=CherryBlossomNETBIOS HN=NoneMAC=None

IP=192.168.1.20DNS HN=CherryBlossom

NETBIOS HN=NoneMAC=None

IP=192.168.1.5DNS HN=None

NETBIOS HN=NoneMAC=00-0C-29-AD-9B-44


During the first week’s assessment, the server colored in red in the above diagram was operating on the network at IP address 192.168.1.5. The example assumes that prior to the second week’s as-sessment an IT administrator changed the red server’s IP address to IP 192.168.1.20 and in addition, changed the yellow server’s IP to 192.168.1.5. The second week’s assessments then proceed. Because of this normal IT network churn, and due to the host tracking limitations present within most exist-ing VM vendor solutions, the VM system mismatches the yellow asset currently using 192.168.1.5 to the red asset. Further, let’s assume that during the first week assessment, all vulnerability tests were executed and the server located at 192.168.1.20 was found to be vulnerable to an SQL Injection and the server at 192.168.1.5 was not found vulnerable. Let’s also assume during the second week assess-ment, only one vulnerability test was executed against the given assessed assets. This is normal because many times an organization only wants to test for the presence of just a single vulner-ability across the network because the scan completes far more quickly. Given this, the most recent assessment can only draw a conclusion about the presence or absence of the one vulnerability it tested for. In order to correctly account for all weaknesses that have not been tested for within the second week, the weaknesses found during the first week assessment must be consulted.

This requires the VM to correctly match hosts assessed within one assessment to how they ap-peared in previous assessments. I refer to this capability as “host reconciliation” but some com-petitors refer to it as “host tracking”. Essentially, it is core capability within all VM solutions which relates point-in-time scanned endpoint findings, with their “real world asset” counterparts. All VM products on the market have this capability, but most of these solutions assume the underlying network is relatively static and as such, are unable to reliably perform this matching in the presence of regular IT network churn.

Several different “host tracking” algorithms of the largest VM providers in the market are described in the referenced whitepaper. However, and as in our example scenario, since IP 192.168.1.5 was in-correctly matched to the red asset, the vulnerabilities associated to the red asset are returned to the SIEM. Assuming there are no overlapping weaknesses between the red and yellow assets, the SIEM will mistakenly conclude that the imminent attack will not be successful. In reality however, the yellow asset which had IP address 192.168.1.20 during the first week assessment, was found vulner-able to the SQL Injection. This use case example illustrates a dangerous false negative occurrence! It is a direct result of the host tracking limitation found in most VM systems on the market.


HOW THIS LIMITATION IMPACTS YOUR SECURITY ECOSYSTEMThe previous use case demonstrates how this host matching limitation found within most VM systems on the market today affects security risk. We may wonder how much impact this limitation has within the security ecosystems of organizations. By using findings revealed as part of a previous whitepaper titled “Vulnerability Management Solutions are Flawed, Leaving your Enterprise at High Risk,” and combined with a few assumptions, we may cast out the effect this limitation has within a security ecosystem across time, in order to gain a better appreciation for these effects. To achieve this, let’s make the following assumptions:

1. An organization is using one of the large VM solutions on the market where assessment- to-assessment host tracking is achieved by one of three network discoverable characteristics: IP Address, DNS Hostname or NETBIOS Hostname, and where the VM administrator may select one of these tracking characteristics which are applicable over a given IP address range. The default tracking mechanism is IP Address.

2. The organization has acquired the VM solution and has been operating it for 1 year but has not maintained it nor understood the consequences of host tracking and therefore has not set the host tracking mechanism. The VM system is therefore tracking based upon the IP address for all assets.

3. The organization has 120,000 endpoints and is scanning these periodically (e.g. weekly).

4. The organization has 40,000 Servers, 60,000 Client machines (desktops and laptops) and the remainder 20,000 endpoints consist of various devices (firewalls, routers, printers, etc.)

Given these assumptions, after three months, the VM solution would have mismatched the Server and Client endpoints at the rates as listed in table 1 (for Servers) and table 2 (for Client machines). After three months, because the system uses IP address to track hosts, 4% of Server endpoints - 1,600 Servers - would have experienced a mismatch (host duplication or a mismatch). Client machines would have experienced 36% mismatch which equates to 21,600 Client machine mismatches. If we cast this out over three more quarters, a subset of those which had not expe-rienced a change during the first three months, are also impacted. After one year, of the original 40,000 Servers, 6,026 of these would have experienced a mismatch at some point in time. Further,


TABLE 3: Endpoint Mismatch After 1 Year

of the 1,600 servers which were mismatched during the first quarter, 4% of these, or 64 change over the second quarter. Similarly, after one year, of the original 60,000 Client machines, 49,934 experienced a mismatch. These findings are summarized in the following table.

ASSET TYPE NUMBER ASSETS MISMATCHED AFTER 1 YEAR % MISMATCH

Servers 40,000 6,026 15.1%

Client Machines 60,000 49,934 83.2%

The following figure revisits our security ecosystem of figure 1, with the VM dirty data in context.


INCIDENT RESPONSE

IAM

SIEM

IAM

GRC

DLP

IPS

After 1 Year

15.1% Server Mismatch

83.2% Client Mismatch

FIGURE 5: Security Ecosystem VM Dirty Data After 1 Year Time


SOLUTIONThe DDI Frontline Vulnerability Manager (FVM) system offers two types of assessment scanning methods; remote unauthenticated network discovery as well as authenticated discovery. For assessments where authenticated discovery scanning is used, FVM is able to uniquely identify the scanned hosts and map these to real world assets. For assessments where remote unauthenti-cated discovery scanning is used, FVM uses a patent pending algorithm in order to map the hosts within an assessment, to the assets FVM discovered during past assessments which represent an organization’s real world network assets. Analogous to methods which match scanned fingerprints to previously stored fingerprints in a database, the FVM Host Reconciliation algorithm compares over 20 remotely discoverable host characteristics for each assessed host, and compares these versus the characteristics previously discovered and stored for the organization’s real world assets.

This virtually eliminates the asset mismatch problem previously discussed and which is present with other VM solutions.

FVM easily integrates with other vendor security solutions, as well as with any element within an organiza-tion’s security ecosystem, by way of an Application Programming Inter-face (API) known as FVM API Quick-Connect. This API uses a standard RESTful interface, allowing both technology, and users, to pull Front-line’s purified asset and vulnerability information.


CONCLUSIONThere is significant opportunity to achieve high value from various integrations of traditional siloed information security solutions, including integrations involving VM solutions. However, VM solu-tions are challenged in relating assessment information to real world assets over time. The accura-cy of one single assessment is very important and remains a benchmark traditionally used to com-pare various VM solutions and products. However, in today’s mature VM market, scan accuracy does not vary significantly from one vendor to another. Unfortunately, far less attention has been paid to the accuracy of these results as they relate to comparison of scan results over time. The reality is that the findings portrayed within the “asset views” of the VM systems used by most organizations (including the Fortune 500 Enterprises) are far less accurate than we once believed. Organizations use this inaccurate information to guide their security decisions, and integrate it with their security enforcement technologies within their security ecosystem. The reason this data is inaccurate is the inadequate techniques used by most VM solutions in dealing with the crucial matching chal-lenge as described extensively in this paper as well as in the referenced whitepaper. The results of the Network Host Reconciliation study summarized in this paper show clearly that hosts which are part of organizations’ networks are actually subject to significant and frequent change, illustrating the potentially severe negative impact these inadequate techniques introduce, seriously impairing the usefulness of integrated solutions within the security ecosystem and which source from VM solutions.

The primitive algorithms found within the inner working of the VM solutions, supplied by even the largest of the vendors in the space are seriously limited, and cannot correctly track the findings for the ongoing assessed hosts in the presence of the dynamic network change our study exposes. As a result, an organization using such solutions must take extreme care, not be misled by the risk profile portrayed by these products, and instead must question the matching technology used within these platforms and take action to avoid the pitfalls these offerings present (mainly a false sense of security or the chasing of phantom problems). Organizations who find their current solution is inadequate and whose VM suppliers cannot/will not solve this challenge should either procure a replacement solution that employs more robust reconciliation methodology or, alterna-tively, integrate with a third party solution which can detect and eliminate the errors present in the information provided by their current VM solution. Users who do neither will, unfortunately, experience negative impacts within their security ecosystem as illustrated in this paper, and even worse, be unaware of the serious security risks present.

is your security ecosystem inaccurately portraying your ...a digital defense, inc. white paper is...

Documents