Cyber Security - Key Performance Indicators

The 2019 EY GISS  (Global Information Security Survey) speaks of three fronts that organizations need to progress on.

Protect the enterprise:  Focus on identifying assets and building lines of defense.

Identifying crown jewels and implement appropriate protections mechanisms to detect and prevent breaches. Most large enterprises have a heterogeneous mix of technology ranging from Mainframes, Wintel, Unix/Linux, Virtual  and now cloud environments; maintaining an accurate IT Asset inventory that includes Hardware, Applications, Databases, Storage etc. is a challenge for most CIOs and CTOs. A virtual server could be spun up in a matter of minutes and spun down just as quickly. Building processes and implementing tools to manage, monitor and report on compliance is an important factor in protecting enterprise assets.

Optimize cybersecurity:  Focus on stopping low-value activities, increasing efficiency, and reinvesting the funds in emerging and innovative technologies to enhance existing protection.

Companies are moving towards automation, data analytics and AI to increase efficiency however a misconfigured tool could lead to a breach going undetected. Many Advanced Persistent Threat (APT) incidents are a result of broken processes and/or misconfigured devices and tools. FireEye reports  that the mean dwell-time for 2018 in the Americas is 71 days, EMEA is 177 days and APAC is 204 days. This allows attackers a significant amount of time to go through the attack cycle, propagate and achieve their objective.

Enable growth: Focus on implementing security-by-design as a key success factor for the digital transformations that most organizations are now going through.

Cybersecurity is becoming a vital part of corporate strategy in 40% of enterprises surveyed board members are now taking responsibility for cyber security as the success of digital transformation initiatives

In the PwC Digital Trust Insights  survey, just 27% of respondents who said that they are comfortable that the board receives adequate reporting on cybersecurity and privacy risk. An effective monitoring and reporting process enables Management and Leadership make informed decisions.

Cyber Security Metrics and Measures from NIST has the following to say:

"Cyber security metrics and measures can help organizations

• verify that their security controls are in compliance with a policy, process, or procedure;
• identify their security strengths and weaknesses; and
• identify security trends, both within and outside the organization’s control.

Studying trends allows an organization to monitor its security performance over time and to identify changes that necessitate adjustments in the organization’s security posture. At a higher level, these benefits can be combined to help an organization achieve its mission by

• evaluating its compliance with legislation and regulations,
• improving the performance of its implemented security controls, and
• answering high-level business questions regarding security, which facilitate strategic decision making by the organization’s highest levels of management."

These are attributes that I consider useful when developing Key Performance Indicators (KPIs). 

For example, in the context of vulnerability scanning:

• Accuracy - The number of false positives on scan reports? 
• Timeliness - Have identified Critical, High & Medium issues been remediated within defined timelines?
• Completeness - Have systems on the network that should have been scanned have been scanned; Compare CMDB/Asset Database and Scan population;
• Authorization - Failed credentialed scans

We can create as many metrics as there are data points however, do a sanity check and drop KPIs that fail the 'what does it mean?' test.  I have come across dashboards that provide inaccurate and sometimes conflicting messages.

In no particular order, here are some metrics should be on every CISO's dashboard.

 Identity & Access Management

 Identity & Access Management:

• Time taken to deactivate credentials
• Entitlements review: Number of users with excessive entitlements
• Third party (Suppliers/Vendors/contractors) access review: # third party unnecessary access removed
• Percentage of employees with Privileged access who are monitored:  monitoring users that have 'keys to the kingdom' (super-users) provides insight to determine if too many individuals have unlimited network access and restrict access to those who absolutely need it.

Configuration Management

Configuration Management:

• % Servers and devices compliant to hardening standards - configuration drift is a risk as IT environments undergo changes, with the widespread adoption of Dev-ops, changes could occur many times daily.
• Firewall/switch audit results
• Number of unidentified devices on corporate network (Wired & wireless)

Security Awareness

 Security Awareness:

• Training compliance levels- % Completed
• Results of phishing and other social engineering tests on staff: % Failed

Security Incidents

Security Incidents:

• Number of incidents detected (SIEM/AV/Malware etc.)
• Mean Time to Detect- How long does it take for the security team to become aware of a threat? According to the Cyber Evolution: En-Route to Strengthening Resilience in Asia-Pacific report, the median number of days between network intrusion and the detection of the threat actor on a global scale is 99 days in 2017. 
• Dwell Time - The time elapsed between the Detect and Resolve phases.
• Mean Time to Resolve - How long does it take to remediate?

Compliance

Compliance:

• Open Audit issues
• Open exceptions /ageing
• Regulatory compliance status (FED, OSFI, SEC)
• Industry Standards Compliance (PCI)

Data Leak Prevention

Data Leak Prevention:

• Data classification levels: If we are blind to our sensitive data sources DLP will fail. This metric indicates the number of databases, devices, endpoints, file shares with no controls.
• Database fingerprinting levels: Databases holding sensitive data must be fingerprinted and available to the DLP tool. This metric gives an indication of the risks associated with databases which are yet to be fingerprinted.
• Number of un-managed devices with sensitive data: Ideally all devices/endpoints should be visible to the DLP tool.
• Number of incidents reported by LOB
• Number of policy exceptions by LOB

Vulnerability and Patching

Vulnerability and Patching:

• Number of systems with known critical and high vulnerabilities: While reporting on all systems is the norm, I prefer to have management reports that focus on the high risk systems and applications (crown jewels)
• Patch levels of High risk systems with known critical and high vulnerabilities: This gives an indication how effective the patching cadence is
• Number of systems with critical and high vulnerabilities that vendors have not released patches yet – alternate mitigation measures applied or if  no mitigation is possible, accept  risk
• Time taken for vendors to release patches
• Days to roll out patches from vendor release

This is by no means a comprehensive list, I would like to hear from you and add on to this list of KPIs.
Please leave your comments and feedback below.