Control Maturity vs. Control Risk: A Client Discussion

A client for whom I serve as CISO advisor posed an interesting question to me last week, “What if we measure and report on control maturity instead of risk?”

A productive and interesting discussion on the topic ensued over the next forty-five minutes. I had never before received this question, so I had to literally think on my feet (thank you Toastmasters!). The following is a rough account of the thought process we went through on this topic.

Measuring maturity is a good thing for my client’s organization, as their maturity is low in many instances.

Measuring the maturity of controls overlooks any risks that may not be addressed with a control. In other words, controls may have (or be approaching) desired maturity levels, but what if there is a real threat for which no control exists? This could be a problem.

I next postulated a blended approach: measure risk but report on controls maturity. But I quickly discounted that idea as I could not see a valid correlation between risk and the maturity of any control. Instead, I described an auditor’s objective when examining a control: its effectiveness. Does the control operate as designed? This is important to know—as important, or even more important, than the control’s maturity.

Then I had an epiphany: You could have a mature control that is ineffective and does not address relevant risks. My client was intrigued and asked for an example.

Thinking on my feet again—in a few moments, I had a good example: An organization has a traditional anti-virus product with a centralized console that provides visibility and control. Console operators can quickly see which endpoints are working correctly, which are not, and which are not even covered. Operators are alerted when viruses are detected on endpoints. There is daily, weekly and monthly measurement, executive dashboards, and changes are occasionally made to improve things. The problem is, some of the traditional anti-virus products are largely blind to current generations of malware, which deliver unique payloads to each infected machine. So this is an example of a highly mature control that is all but ineffective. Nail in the coffin for looking only at controls maturity.

I returned to controls maturity and we discussed it a little more. We reasoned that a more mature control is one that is being watched: it has a formal design, and it’s being measured, monitored and improved. This is indeed a good thing. Still, no organization (this one, anyway) needs to be at the Capability Maturity Model (CMM) level 5 across the board. 

The discussion with my client came back to risk. They agreed that we could not throw out the risk baby with the bathwater. Risk is important, and we need to keep our eye on it, including being open to the possibility of new and changing risks over time. This is classic risk monitoring. However, we can understand where the greatest risks are located within the business, and then make sure controls in those areas are effective and have a maturity level that is commensurate with levels of risk.