Security is hard, and when something is hard we seek shortcuts. Such standards imply such a promise. They can be very prescriptive, telling you exactly what to look for: make sure you have an incident response plan, make sure you use random keys, make sure you use AES, make sure you are made aware of new exploits that are discovered, etc.

The pros

Should such standards or checklists be used? — Absolutely. Why? — Because it is like a second pair of eyes looking over what you do and making suggestions. Whereas they are not perfect, they serve at least those two purposes (other than allowing you to claim compliance with something):

• they provide you with a way to know that you have not missed anything, by going over the list and making sure that each bullet is either handled or is inapplicable, and

• in organizations that do not have dedicated skilled security personnel, they assure the organization has something, which is way more than nothing.

 The cons

There are two primary difficulties with security standards and checklist which are prescriptive. First, they often do not cover every aspect needed, and second, they attempt to impose a prescriptive approach where such cannot be fully applied.

Scope

Depending on what the system you are trying to architect is, you may notice that it is of a wider scope than that of any standard. At the highest level, there are two distinct types of system security:

• security engineering, and

• secure operation

The first involves implementing the system correctly; not deploying — implementing, as in writing source code. This discipline is concerned with logical flaws, coding flaws, proper use of cryptography, etc. The second deals with securely operating a system that has already been engineered. This discipline is concerned with network security, firewalls, intrusion detection, incident response, denial-of-service attacks, etc. Those two disciplines have barely anything in common, and so even the standards that cover them are separate. For example, FIPS 140 is targeted at the first discipline whereas NIST CSF targets the second one.

Coverage

Checklists and standards are prescriptive in nature, that is, they tell you what to do, or where to look. Security is a scientific art that requires discovery, and discovery cannot be prescribed. I am urged here to quote from my own post, On the Purpose of Security Standards:

It is very difficult, and in most cases impossible, to write a checklist that assures the security of a complex system. For a checklist to be “checkable” it must be drafted positively, as opposed to a security policy/goal which is usually drafted negatively. A checklist includes states that shall happen, whereas a security goal is a set of states that shall not happen. Mapping requirements of the second type to requirements of the first type is difficult.

This quote also references: The toughest part of designing secure products.

Security engineering is all about anticipating attack vectors and setting the requirements for mitigating them. This work requires discovery, which cannot be prescribed verbatim. A checklist can certainly help: it can tell you where to look, it can hint you on how to think, but it cannot make you discover, just as it cannot make you invent.

Security standards are good references, and they can point you to areas you may have overlooked. Just don’t let them derail you off the important part of the work, which is the adversarial mindset view of your own system.