Vendor Contract Checklist

Don’t forget to include a review of your contracts with vendors when reviewing your own security compliance. Here is the short version of my checklist for vendor contracts:

  • Confidential Information – How can the vendor use confidential information?
  • Safeguarding Information – Do you require the vendor to meet specific standards, have specific controls, keep data within the borders of the U.S.?
  • Oversight – When and who does the audit of the vendor?
  • Data Breach Procedures
  • Compelled Disclosures – Are they required to tell you in time for a legal response to subpoenas?
  • Termination Procedures – What do they do with information when the contract ends?
  • Subcontractors – Are subcontractors of the vendor required to meet the same standards?
  • Employee training
  • Insurance and Indemnity requirements
  • Definitions – Do you have definitions that match your requirements? For example, does your insurance definition of breach match your contract definition?

5 types of “security audits”

We are often told by clients that they want a security audit. The word “audit” is often inappropriate. Here are five types of security evaluations and their goals and audiences:

1) Vulnerability Tests
2) Penetration Tests
3) Risk Assessments
4) Compliance Audits
5) Due Diligence Questionnaires

Vulnerability Tests
A vulnerability is a “flaw or weakness in system security procedures, design, implementation, or internal controls that could be exercised (accidentally triggered or intentionally exploited) and result in a security breach or a violation of the system’s security policy.” Vulnerability tests check for the existence of known flaws. Here are some useful characteristics of vulnerabilities:

1) There is a fix
2) There is a no fix
3) There is an exploit available
4) There is no known exploit available

Vulnerabilities for which there is no fix and there are known exploits are causes for immediate mitigation. The vulnerability tester may provide a list of mitigations as part of their analysis. A vulnerability test ought to be done before a system goes live to ensure that no flaw exposes the systems despite security controls. Vulnerability tests can be performed externally—inside or outside the organization’s firewalls—or on the system itself. The results will be significantly different. A vulnerability test installed on and performed against the system on which it is installed will produce results that are much more extensive then the same test performed from the network or outside of the firewalls. Even the credentials used to perform the vulnerability test will affect the results. Patch updates are sometimes followed by vulnerability testing to ensure that security patches accomplished their task. Choosing where to test from and what credentials to use during the test is an important part of what the tests are intended to show. For example, if you want to show just how far an insider can get when attacking, use an insider’s credentials and test from inside the network or from approved remote access. On the other hand, if you want to show what a random Internet attacker can see, test from the Internet and without any special credentials.

GOAL: The goal of vulnerability tests is to show what still needs to be corrected to meet the organizations security controls.

AUDIENCE: Executive management may only want to see pass/fail results. Technical staff wants the details so they can correct. IT Management needs the results both to show they are doing their jobs and to check on their own staff and vendors performing the work.

Once a vulnerability is known, the organization is on notice about it. This means that if there is some resulting damage because of failure to mitigate, then the organization can be liable for the damage. Don’t refuse to have the test for this reason though. There is similar liability for “should have known”.

Testers might be violating their terms of service of their own Internet connecting by performing the tests. Be sure that that the agreement with the tester places liability on them for any actions by their own Internet providers for performing the test.

Be sure that testers are clear on which systems and networks to test. Some tests can lock or reboot a system and if this is done to someone else’s systems at your request then your organization becomes liable. There is some protection if the tester is an independent contractor.

Penetration Tests
Penetrations tests are attempts to breach existing security. The scope of a penetration test can vary. Sometimes the test is for a single application on a single server. Or the entire perimeter defenses are tested to see if they are protecting assets adequately. In addition to purely electronic penetration tests, social engineering may be employed. Social engineering is a method of deceiving someone. The goal is to convince someone to perform a task that helps in attacking the organization. This may be as simple as opening an email with malware attached. It may be by getting an inside person to make changes for the attacker. In a large organization, social engineering attacks are almost always successful. People are just too easily fooled by those experienced in doing social engineering attacks. In contrast to vulnerability tests, a penetration test is also a test of the skill of the tester. It is a lot more difficult to accomplished the actual breach than to simple check if there are vulnerabilities that would permit a breach.

GOAL: The goal of a penetration test should be to show whether a reasonable attack against your organization will succeed or not.

AUDIENCE: Executive management may only want to see pass/fail results. Technical staff wants the details so they can correct. IT Management needs the results both to show they are doing their jobs and to check on their own staff and vendors performing the work.

Risk Assessments
Risk assessments are used to identify, estimate, and prioritize risk to organizational operations (i.e., mission, functions, image, and reputation), organizational assets, individuals, other organizations, and the Nation, resulting from the operation and use of information systems. The purpose of risk assessments is to inform decision makers and support risk responses by identifying: (i) relevant threats to organizations or threats directed through organizations against other organizations; (ii) vulnerabilities both internal and external to organizations;(iii) impact (i.e., harm) to organizations that may occur given the potential for threats exploiting vulnerabilities; and (iv) likelihood that harm will occur. The end result is a determination of risk (i.e., typically a function of the degree of harm and likelihood of harm occurring). Risk assessments can be conducted at all three tiers in the risk management hierarchy—including Tier 1 (organization level), Tier 2 (mission/business process level), and Tier 3 (information system level). At Tiers 1 and 2, organizations use risk assessments to evaluate, for example, systemic information security-related risks associated with organizational governance and management activities, mission/business processes, enterprise architecture, or the funding of information security programs. At Tier 3, organizations use risk assessments to more effectively support the implementation of the Risk Management Framework (i.e., security categorization;
security control selection, implementation, and assessment; information system and common control authorization; and security control monitoring).

GOAL: The goal of a risk assessment is to let management know what risks exist so that they can decide if they are willing to accept those risks or if they should invest time and money to reduce the risks.

AUDIENCE: Executive management and the board are the only ones who can really accept risk for the entire organization.

Compliance Audits
Every organization has some set of rules that must be followed. Compliance audits are intended to see if those rules are followed. From a security standpoint, which rules to test against make an enormous difference for the compliance audit. It would be almost useless to simply pick a standard and check for everything unless your organization has been through the process of setting security controls in place prior to the audit.

GOAL: The goal of a compliance audit is to show a pass/fail result of how well the organization meets the required rules tested against.

AUDIENCE: Executive management and board of directors needs assurance that their legal obligations are being met. IT Management needs to know if their controls are holding.

Due Diligence Questionnaires
Due Diligence questionnaires are a subset of compliance audits. Often, an organization does business with others that require them to show, for their own compliance, whether their partners meet or exceed the security controls they have. Every organization should be doing this with its vendors. If you handle the confidential information of another business then you may have seen due diligence questionnaires. Before creating questions for vendors, the standard of security must be chosen.

GOAL: When answering due diligence questionnaires, the goal is to answer truthfully but also not to divulge confidential information about security. Fudging on the answers puts the organization at serious risk. It’s better to answer truthfully or not at all then to mislead. When creating questionnaires for other organizations, the goal should be to ask the kinds of questions that show whether they can maintain the confidentiality, integrity and availability required by existing security policies. To do that, you need to have already set your security goals and be on the way to meeting them yourselves.

AUDIENCE: Compliance auditors, Executive and IT management.

Joel Colvin has been a security consultant since 1992 and an attorney since 2015. If you would like to know more or have a version of this presentation at your organization, please contact him at

After the Breach: Legal and Technical Issues

Long before it actually happens, every organization should prepare for when their networks are breached. Do you even know what you have to do? This presentation will discuss legal notification requirements and some of the technical solutions that reduce the reporting requirements and protect your firm. This discussion is intended to familiarize CIOs and staff with the legal issues before their firm lawyers ever get involved. We will cover:

1) Factors in deciding to act for litigation or solely for recovery
2) What kinds of internal investigations are protected from discovery in litigation and more importantly, what kinds are not.
3) Who can and should do your data forensics
4) Existing breach notification in Texas, the rest of the United States, and the world.
5) The trend in breach notification
6) Non-breach required notifications in Texas.

Joel Colvin has been a security consultant since 1992 and an attorney since 2015. If you would like to know more or have a version of this presentation at your organization, please contact him at

Information Classification Should Drive IT Planning

Information classification is an integral part of implementing an information security framework and performing risk assessments. Proper classification leads to the selection of appropriate controls. When the goal of information security is to protect, how can this be done without knowing what value differing information types have to the organization? What’s more, information classification can be the method to trigger technology planning for the whole organization well beyond the selection of security controls.

One of the beauties of classifying things is to give us a shorthand method of understanding. For those of us weaned on the OSI seven-layer model, when we are introduced to a new networking protocol we can place it in its appropriate layer by asking about the defining characteristics. Once we know where this new protocol belongs in the model, we can make other fairly accurate assumptions about what the protocol must do, what it cannot do, how it can be exploited or protected.

Information classification works the same way. If you know that a piece of information is sensitive and should not be shared with the public, then you know, in a general sense, how that information should be protected, what kind of access controls should be in place, and that disclosure might trigger some damage to the organization. This kind of broad knowledge about data allows organizations to clean up existing systems that handle information as well and select and build replacement systems or third-party services that contain this sensitive data. Everything from internal controls, vendor contract clauses, selection of types of services, and methods of transport are all impacted by the knowledge that sensitive information is involved.
Unfortunately, too many organizations operate without planning based on how information types should be treated. Instead, they deploy systems and controls for one dataset at a time. Classifying types of information in advance can save time, jobs, and be a driving force for IT planning. Classification helps provide better service to information stakeholders by aligning their needs with the implemented protections.

Joel Colvin has been a security consultant since 1992 and an attorney since 2015. If you would like a copy of the complete paper or would like help in developing your organization’s security policies, please contact him at