TL;DR: SOC2 is the most requested security certification for B2B SaaS, and it does not test whether your systems can actually be compromised. Auditors verify that controls exist and were operated — they do not attack your systems, validate control effectiveness, or find vulnerabilities. Understanding these gaps is essential for building a security program that goes beyond compliance credibility to genuine security assurance.
When a prospective enterprise customer asks "are you SOC2 compliant?" they are asking a proxy question. What they actually want to know is: can I trust this company with our data? SOC2 has become the standard answer to that question in B2B SaaS — but the question and the answer are addressing fundamentally different things, and the gap between them is where breaches happen.
SOC2 is an attestation standard governed by the AICPA. Its purpose is to provide assurance that a service organization has implemented controls relevant to the Trust Services Criteria — security, availability, processing integrity, confidentiality, and privacy — and that those controls operated effectively over the audit period. It is an audit of process and control consistency. It is not a security assessment, and understanding precisely what auditors do and do not do is critical for anyone building or evaluating a security program.
What SOC2 Auditors Do Not Do
They Do Not Run Penetration Tests
SOC2 CC7.1 requires that organizations implement monitoring controls to detect potential threats and anomalies. A penetration test is commonly accepted as evidence that such monitoring and detection controls are being applied. But the auditor verifying CC7.1 compliance looks for documentation that a pentest occurred — they review the report, note the scope and findings, and may ask about remediation. They do not conduct any testing themselves.
This distinction matters enormously. An organization can satisfy CC7.1 with a narrowly-scoped penetration test of a staging environment that bears little resemblance to production, conducted by a provider with minimal methodology rigor, producing a report with only medium-severity findings because the scope was too narrow to expose critical risks. Auditors will not identify this as inadequate. They will check the box.
They Do Not Test Control Effectiveness
SOC2 auditors test whether controls were in place and operated during the audit period. They use sampling — reviewing a subset of evidence across the audit period to verify that controls were consistently applied. They are not testing whether those controls actually prevent a motivated attacker from achieving their objective.
A control that says "access to production is restricted to authorized personnel with MFA" will be audited by reviewing the access control policy, inspecting the list of users with production access, and verifying MFA is configured. It will not be tested by attempting to bypass MFA, by attempting privilege escalation from an authorized account, or by exploring whether the list of authorized users is actually accurate. The control could be fundamentally weak — MFA using SMS that is trivially phishable, an access list that includes inactive accounts — and still satisfy the audit requirement.
They Do Not Find Vulnerabilities
Security vulnerability discovery requires active interrogation of systems — probing for injection flaws, testing authentication mechanisms under adversarial conditions, analyzing business logic for bypass opportunities. SOC2 auditors conduct interviews, review policy documents, inspect configurations through screenshots and exports, and sample evidence logs. This approach is appropriate for an audit; it produces no security findings.
An organization can complete a SOC2 Type 2 audit with an unpatched critical vulnerability in their authentication layer, an exposed API endpoint with no rate limiting, or a storage bucket containing customer data with public access enabled — and none of these would be identified by the audit process.
They Do Not Technically Validate Cloud Configuration
Cloud security misconfiguration is one of the leading sources of data breaches for SaaS organizations. SOC2 auditors assess cloud configuration primarily through policy documentation, screenshots provided by the auditee, and configuration exports. They do not conduct independent technical verification of cloud configuration state.
This means that an organization can represent their AWS configuration as compliant with their security policy in audit documentation while the actual configuration diverges — whether through configuration drift, human error, or deliberate minimization of the disclosure scope. Independent technical assessment — cloud security posture management tooling, cloud configuration review, or cloud-focused penetration testing — is the complement that SOC2 audits do not provide.
They Do Not Verify Your Vendors' Security
SOC2 requires that organizations have a vendor management program — documented processes for evaluating and monitoring the security of third-party vendors. Auditors verify that this program exists and that you have a vendor inventory and risk assessment process. They do not independently verify the security posture of your vendors. If one of your critical sub-processors has a material security vulnerability, your SOC2 audit will not identify it.
They Do Not Test Your Incident Response Capability
SOC2 requires documented incident response procedures and evidence that the procedures have been exercised — typically a tabletop exercise conducted during the audit period. Auditors verify the documentation and the exercise evidence. They do not conduct a live incident simulation, they do not evaluate your detection capability against real attack patterns, and they do not assess whether your team can actually execute the plan under pressure.
The Audit Period Can Be Managed
SOC2 Type 2 covers a defined audit period, typically 6 to 12 months. Controls must be operational during that period. But there is nothing in the standard that prevents controls from being excellent during the audit period and degrading immediately after. Organizations under budget pressure sometimes maintain audit-required tooling, training, and processes through the audit period and then reduce investment once the report is issued. This gaming of the audit period produces clean audit reports for organizations that may not have sustained security programs.
The Audit vs. Assessment Distinction
The conceptual distinction between an audit and an assessment is fundamental and often lost in enterprise security conversations. An audit verifies compliance — it answers the question "did you do what you said you would do?" An assessment evaluates effectiveness — it answers the question "does what you're doing actually work?"
SOC2 is an audit. Penetration testing is an assessment. Red team exercises are assessments. They serve different purposes and produce different kinds of evidence. Organizations that treat audit evidence as security assurance are conflating compliance credibility with actual security posture, and this conflation is one of the most dangerous patterns in enterprise security.
Lorikeet Security's assessment services are designed specifically to complement SOC2 and ISO 27001 audit programs — testing the real effectiveness of the controls that auditors verify exist. See our service areas for how penetration testing, cloud configuration review, and attack surface management fit alongside your compliance program.
| Dimension | SOC2 Audit | Penetration Test | Red Team Exercise |
|---|---|---|---|
| What it measures | Control existence and consistent operation over audit period | Real exploitability of systems and controls under adversarial testing | Full attack chain capability against objectives using TTPs of real threat actors |
| Who performs it | CPA firm / licensed auditor | Security firm with penetration testing practitioners | Advanced offensive security team with threat intelligence integration |
| Methodology | Sampling, inspection, interviews, evidence review | Active technical exploitation attempts, manual and tool-assisted | Objective-based attack simulation, evasion, persistence |
| Finds real vulnerabilities | No | Yes | Yes, with business impact demonstration |
| Output | Audit report with auditor opinion on control effectiveness | Technical findings report with severity ratings and remediation guidance | Attack narrative, detection gap analysis, executive summary |
| Board / investor value | High — compliance credibility with customers | High — evidence of proactive security testing | High — most rigorous security assurance available |
| Frequency | Annual audit cycle | Annual minimum; more frequent for high-change environments | Annual or bi-annual for mature security programs |
| Cost range | $15K–$40K audit fees | $10K–$50K depending on scope | $50K–$200K+ depending on scope and duration |
What Real Security Looks Like Alongside SOC2
SOC2 is not valueless — it is a meaningful compliance signal that tells your customers you have a structured security program with audited controls. The problem is when it is treated as sufficient rather than as one component of a broader security program.
Organizations with mature security programs treat SOC2 as the compliance floor and layer actual security assurance on top: annual penetration tests with full production scope, continuous vulnerability management, cloud security posture monitoring, threat-informed detection engineering, and regular incident response exercises that go beyond tabletops. The SOC2 report satisfies the customer's compliance requirement. The actual security program protects against real threats.
The remediation work generated by penetration tests also directly strengthens your next SOC2 audit. Findings discovered and remediated during a pentest represent evidence of proactive security management and control improvement — exactly what auditors and enterprise customers want to see. Book a consultation to understand how to structure your security program to get value from both your audit and your assessment work.
SOC2 Proves Your Controls Exist. A Pentest Proves They Work.
Complement your SOC2 compliance program with penetration testing that validates what your auditors can't. Lorikeet Security conducts production-scope assessments that find what matters — before attackers do.
