The Truth About Audits

This post talks about the good, bad and ugly that we see around cybersecurity audits. It is informed by 4 years working for a company deeply embedded with PCI Compliance, and then about 8-10 years of experience helping dozens of companies with SOC 2, ISO 27001, NIST 800-171A and other similar audits.

The TL;DR is that audits can be helpful for improving an organization's security posture. On the other hand, an audit doesn't prove security and the more you game the audit the less likely it is that the audit improves your security. Furthermore, unfortunately, most audits we have seen are inconsistent across auditing firms and don't mean what you might think they mean.

The Good

Audits force an organization to do some introspection. If you identify appropriate "controls" to check, then the audit activity can help you to make sure you are doing the right things.

For example, if you say you have endpoint protection in place then the auditor may ask for proof. While producing the proof, you may realize that Mac and Linux devices were not actually covered by the tool you had in mind. That might lead you to use a solution that works with those devices, reducing your actual risk materially.

In some cases, the audit introduce controls you didn't know you needed to have in place! This can be extremely helpful for organizations that haven't oriented to a standard (see: why we use NIST 800-53 or which security standard should I use? for more information) and may not have identified broad sets of controls they want to have in place.

Sometimes an audit just tells you what you need to know, but from an outside party. Well, if the respected expert XYZ firm says we need to do it, then we can justify the expense to upper management. This just happens sometimes.

So in some ways, audits can really help organizations improve their security.

The Bad

For every customer we have that has had good experiences with audits, we have at least one that has had bad experiences with audits. That's < 50% positive feedback.

One huge problem is defining the audit Scope. I've seen audits where the auditors review all of the internal corporate IT as if internet facing applications didn't exist and sign off on an audit because the correct controls were in place for internal email and file sharing.

I've seen firms scope an audit a certain way and then come in and bleed customers after an initial audit agreement based on their lack of alignment to the controls.

I've also seen the opposite, audit firms that come in and charge a very low fee and basically check nothing.

Another problem with audits is that most of them aren't pass / fail and nobody reads audit reports. This means that so long as you can produce an audit report (eg. SOC 2), you can usually win business, even if the report says you didn't cover the main application the reviewer cares about or even if you had significant findings. Only very invested companies with strong teams are able to do this justice - and even then, they are subject to the whims of the auditor.

The Ugly

We have worked with a number of audit firms. You might be surprised at how inconsistent the process is. Some firms are quite technical and understand what controls fit in a given environment. Other firms have a list of controls and they are going to stick to them no matter what. Maybe the worst thing is that we've even seen significant variation in experience with different teams from the same firm!

Did you ever stop and think about how some of the most famous hacks were conducted against organizations that had been audited? Consider Target and PCI. Consider almost any recent hacking incident and SOC 2 Type 2. Was Heroku SOC 2 audited? GitHub? Okta?

It reminds me a bit of when I worked for a security company that helped build the PCI Compliance standard. I invite you to think for a moment about how good a business it is to define the standard for compliance, get the card brands to agree to your standard, then define the certification process for assessors, then be the main company that gates compliance to that standard! Can you say boondoggle!? But even PCI was at least a little bit prescriptive.

There is also a growing industry of tools to accelerate compliance. They typically integrate with a bunch of data sources and consolidate information. Whenever I see this, and the idea of automating compliance, I have to say - I do not think that works the way you think it works.

I do not think that means what you think it means.
Security and Compliance

For example, you may have a tool that claims to confirm that you are using a password manager. Great. Does it know which passwords are in scope and that those passwords are stored ONLY in the password manager? Can it tell if you are actively using the password manager?

Let's take another example, suppose a tool tells you your AWS environment is SOC 2 compliant. What does that even mean? Does the tool understand anything about what your dataflows, user lists, and other details are? Do they think they can confirm ALL of your security settings are applied properly AUTOMATICALLY? Consider the question of who has access to what data? I've basically never seen a tool in any security domain that can effectively automate that.

So this whole idea that we can automate compliance is either jaw droppingly naive to me or reflects on the inadequacy of the audits - OR BOTH! And by the way, we build security tools that are directly adjacent to (but different from) both audit compliance ( and AWS Compliance ( One thing I will say, I'm all for using API's and open tools like that let you get the data you need for compliance. I will just insist that you have to be able to ask and answer questions and not just run a tool. Some auditors we work with won't accept CLI output as evidence, they need to see the AWS console. How do we reconcile that with a fully automated acceleration tool that is checking a bunch of things that may or may not matter?

So the idea that any audit (and particularly SOC 2) consistently demonstrates security is so obviously far from the truth that I can't believe that "the community" is using it as a standardized baseline these days.

The worst thing is, I don't believe it is ignorance so much as avarice on the part of the auditing community that we see this. Certainly many in the security community know that these audits are fundamentally flawed.


We work with some audit firms that we respect and like a lot. We believe they help our customers improve and hold themselves accountable for their internal security.

We also work with firms and tools that we think don't add a lot of value - let alone security - and which make a lot of work for the organization.

We recommend that you go into these relationships with eyes wide open about what you are trying to get and what your audit actually means based on the price and type of auditor that you have.

That being said, maybe it is time for a new audit standard. Maybe it is time for a better security standard. In either case, we believe collaboration and people will be a foundational part of it.

Phishing Job Candidates

A job candidate received a solicitous phishing email from what looked like a valid client domain but it turned out it was not the client. The call was not coming from inside the house ...

We recently came across a phishing campaign at a client that caught our attention because it was highly targeted to a company (our client) but the targets weren't the typical "internal employees". Rather the targets were outside people that had reason to interact with the company - in this case job candidates and even potential candidates. This introduces some interesting wrinkles into the usual approaches for defending against phishing and protecting organizational reputation, so we thought it would be worth highlighting some of the details of the campaign and what we did.

Phishing is where someone, typically some sort of organized cybercrime gang, sends a malicious email to a group of people hoping that someone will respond, click a link, open an attachment or something like that. The objective is typically to compromise credentials or the user's computer, or potentially to collect secret information (eg. account info) to perpetrate fraud. We have written previously about spearphishing.

What It Looked Like

In this case, the phishers did the following:

From the candidates perspective, the emails looked somewhat realistic.

Dumb Luck Detection

With most phishing campaigns, you see lots of evidence of them. Employees report that they got a weird email. Or maybe you even get the weird email yourself. At our more advanced customers, we have ways of sharing information about new campaigns we see - eg. sharing screenshots of examples in a #security channel.

With campaigns that target potential job candidates, the candidates don't have this avenue for discussing things with the company. Unless a candidate just smells something phishy and decides to tell you about it, how would you find out about it? You certainly can't train the planet to prevent people from falling for phishing related to your organization.

In this case, the only reason the campaign was detected at all was that one of the real candidates (again not an employee!) was also targeted by the phishing campaign and called the two different interview processes out to the recruiter from the actual company.

What Can We Do?

One of the keys to the success of this campaign (well, relative success, we're not aware of anyone actually falling for it yet - but people engaged with it) is that the domain looks credible. To prevent this, it can be helpful to register similar domains, like those with:

Once the domain was registered, a second thing we did was report abuse and ask the DNS provider to disable it. It is unclear how effective or quickly this will be done. (It has not been taken down yet)

Another thing we recommended, but which is very hard for companies to do (and this one didn't), is to publish a blog and social post with the detail of the campaign so that potential targets can find information to defuse the emails they are getting on your website.

It should be noted that there are any number of workflows where phishing like this could be done, not just job search. Vendors, partners, customer engagement, etc. There was a short period of time where we were concerned that there was leakage of candidate information through one of the many third party systems hosting the process. After further review, we don't think that was the case, but it still may be something an organization would want to do proactively to prevent these attacks from being more credible than they otherwise would be.

Finally, it is always a good idea to think through your operational processes and communicate about those early and often with people that are interacting with you. So specifically, you can:


A critical step in this scenario was the recruiter listening to the candidate's input and believing them that something was not right. It turns out that being human and communicating has big benefits.

The follow ups are also important. IT looking at the detail to identify the phishing domains, reporting them, and capturing the detail so that the company knew what the patterns were was important. This allowed them to communicate with candidates and update the information in their posting and their more general communication strategy.

Of course, we can't stop an attack like this from happening, and we can't really be responsible for every misuse of our identity - but being proactive and trying to make it easier for candidates not to get fooled by phishers is worth the effort. If you are a company wondering what to do, you could start by adding this as a risk in your Risk Register.

To me, that is is what is scary about this scenario: there is no obvious way to stop it and there is no real limit to what or who could be targeted. So like with many things in security, we have to live in the grey.

Automated Mass Spearsmishing

Phishing is where someone, typically some sort of organized cybercrime gang, sends a malicious email to a large group of people hoping that someone will respond, click a link, open an attachment or something like that. The objective is typically to compromise credentials or the user's computer.

Spearphishing is where such a campaign is conducted in a more targeted way, typically focusing on specific people with more personalized context that would make the campaign more compelling. Whaling is where spearphishing targets executives (think whale == big fish)! A common spearphishing or whaling objective is to get a financial officer or accounting team member to transfer money or change account details so that payments get misrouted.

What we have been seeing lately are campaigns that are conducted at a larger scale that is likely highly automated, but that also have the context required to be compelling and a request that is possible for many tiers of employees (not just finance execs) to do. We also got targeted directly by one of these, so we can share the detail. Let's do it!

The SMiShing

Smishing is where someone is doing phishing (communications with malicious intent) over SMS or text messages. Our particular text looked like this:


In this case, there are a couple of obvious things to note about the Smish.

  1. It is addressed to Keely and obviously sent to her phone.
  2. It claims to be from Matt Konda, who is the CEO of the company Keely works for.
  3. It is from a phone number that is co-located to Matt Konda's typical location. (Texas)

Now in this case, we're lucky, Keely is on the ball and immediately realized that this wasn't real. It might have been the:

I'm excellent with texts ...

Phisher #1

There are some obvious other tells that we should call out:

  1. It was from a new number that is not where Matt typically communicates from.
  2. It is an unusual communication channel for something important.
  3. The urgency but also unavailability to confirm on a call or via a normal channel is to be noted.

Keely didn't respond, so we can't say for sure what would have happened next. However, we have seen this play out with customers with the exact same text (the "I'm excellent with texts" is hard to miss!) but from the customer CEO to an employee. When the employee responded, the campaign asked the employee to purchase Google Play gift cards.

Note that we have also seen other SMS campaigns and even more classic social engineering campaigns (phishing) to get people's phone number that were later used in an SMS campaign like this.


Based on what we are seeing, either this gang is particularly motivated and have time on their hands to do their research, or there are various layers of automation involved.

My guess is that they are using data from a LinkedIn data breach to associate people to companies, grab the company names, the people names, the phone numbers and emails and be able to formulate a programmatic automated but still targeted (contextual) campaign.

A particular interesting characteristic is using the CEO as protagonist in texts. It is common to see this used when an account has been hijacked to do the same thing, but maybe because not everyone has the CEO's real cell phone number it isn't always obvious that it isn't coming from them? SMS doesn't have the context (eg. signature, logo, etc.) that email does. Then again, with the data from LinkedIn (or whatever it is) the attacker could probably make a fake signature that looks pretty realistic substituting title, role, company, logo, etc.

Note that we have also seen other SMS campaigns that are similar in the sense that they use the CEO role but different types of messages - sometimes even targeting NEW employees.

Of course there are also the standby classic social engineering campaigns (phishing) to get people's phone number that were later used in an SMS campaign like this.


Educating employees about social engineering like phishing and smishing is a key part of a security program and can be one of the most important things you can do. We want employees:

SPIO can help provide this training. One of the customers that was targeted said that it was our training that made them stop and not follow through.

... your training was spot on in triggering all of the necessary awareness for me to start varying this exchange


Your Next(or First) Security Hire Should Be...

For years, a common rule-of-thumb said your security spending should be around 10% of your company’s IT budget—but that rule doesn’t quite hold up anymore. In fact, a 2020 Deloitte survey on cybersecurity says this number is now more like 10.9% and rising year after year. That’s not surprising, as cyberattacks keep getting more sophisticated, and more companies of all sizes get targeted. There may be significant accumulated technical debt for those organizations that have not spent that needed 10% for security over the last few years.

For most smaller companies, that 10 or 11% means you can't hire additional FTE security people until you have at least 200 employees, and even then, you have to be very selective. So, when you’re ready, how should you approach hiring in-house, full-time security personnel? We shared our thoughts on who your first security hire should be here. The TLDR on that is: It depends on a lot of factors, but it should probably be a DevOps person. A skilled DevOps person can code and automate tasks that will help you make the most of the security platform tools that do the heavy lifting of your security program.

One of our clients recently hired several security personnel. They started by hiring a chief information security officer (CISO). They followed that by hiring a security engineer, followed by a governance, risk and compliance (GRC) officer, then an application security engineer, and finally a DevOps person.

That’s a pretty sizable security team for a small company, and it means they’re spending more on security than most companies of their size. Most SMBs and start-ups can’t afford this kind of security team, even if they do ignore the 10% rule. Further, those roles might not even be the types of immediate security hires that makes sense for them.

How you invest resources in security will vary depending on the risks, profile, and priorities of your company. Planning a security hiring roadmap is a bit like growing your security program, and it starts with an analysis of your company’s needs.


When you focus on your risk priorities, you can think broadly about the most effective way to address them. Should you bring on a new hire, outsource to a security service provider, or invest in software tools or external SaaS security platforms?

For example, due to its industry one of our corporate clients is a ripe target for specific types of fraud, including bot automation and account takeover. They hired employees who focus on preventing these types of fraud by mapping application paths and defining new “speed bumps” against these types of attacks. In their case, building in-house security expertise on the threats specific to their business is a smart investment. They can use a combination of outsourcing and security tools to address the more common security issues that all companies face.

Speaking of which, one of the most common security controls all companies need to implement is endpoint security. Because endpoint security is a universal, high-priority security need, it has a well-developed ecosystem of tools and service providers to which companies can outsource this task. Consequently, we usually see small companies either task their existing IT personnel to managing the endpoint tools or outsourcing it to an IT firm.

Another universal risk area is network architecture, configuration, and monitoring. If you have IT personnel with strong network skills and experience, they can use a proper set of network security tools to manage scanning and monitoring the network for vulnerabilities or intrusions. If your first security hire was an experienced DevOps coder, they can write scripts to leverage these tools to improve the company’s ability to detect, analyze, and respond to risks in (and threats to) your network infrastructure. Of course, network management and monitoring can be and often is outsourced entirely. Network monitoring is a 24/7 job, which requires multiple personnel, even when automation is handling the rote and scale tasks. For this reason, outsourcing can be less expensive for a small company than building a 24/7 monitoring team in-house.

The most common scenario is for small companies to use a combination of tools, staff, and outsourcing to meet the full scope of their cybersecurity needs. Another client—one with high privacy requirements due to the nature of the data it handles—leverages the SPIO platform to continually mature its security program. At the same time, it also works with an outside privacy security consultant and assigns task execution responsibilities to an internal DevOps team. Through this combination, the company benefits from SPIO security expertise to grow their security program, while plugging in additional privacy expertise specifically targeted to their industry’s domain.


In our post on your first security hire, we discussed the challenges of balancing senior leadership experience with practical task experience in a more junior role. We stick by our recommendation of starting with practical DevOps experience for your first security hire or two. Their functional expertise means they can leverage both security tools and outsourced expertise to put your company quickly into a strong security posture.

However, you will need somebody in senior management with authority to oversee company IT security. Identifying that senior person is one of the 21 steps your company can take to immediately improve its security posture. Senior IT security responsibilities can initially be delegated to the head of IT, the risk management/GRC officer, or your vendor management team. These employees may not have the practical security experience, but that’s why it is important that your first dedicated security hires do.

While it can be a challenge for a small but growing company, you want to bring on someone with senior IT security experience as early as possible, so your security program develops and operates strategically rather than tactically. Remember that, even if you bring this person into a hybrid security role, their experience enables them to best leverage security compliance management software like SPIO and third-party security experts for a well-rounded security program. As your company expands and starts looking at working with bigger companies with more stringent security expectations, it will work to your advantage to have someone with seniority who can talk confidently with prospects about your company’s security program.