SOC 2 Quality Guild

Practitioner-driven standards for SOC 2 report reliability

A community creating standardized evaluation criteria to help GRC and TPRM practitioners assess how much weight to give a SOC 2 report when making vendor trust decisions.

Read Rubric About

What We Offer

Open resources built by practitioners, for practitioners

SOC 2 Reliability Rubric

11 standardized signals across Structure, Substance, and Source to help you determine how much weight to give a report — before it influences a vendor trust decision.

Tactical Response Guide

Got a low-quality report? Eight practical approaches for working through quality concerns with vendors, stakeholders, and risk owners — without burning the relationship.

Contribute

Vote on community-driven projects — from example libraries and scoring frameworks to expanding coverage beyond SOC 2. Your priorities shape what gets built next.

The Three Pillars

Our framework evaluates reports across three critical dimensions

Structure

Does the report include required components and maintain professional consistency?

Substance

Do the controls, testing, and conclusions logically align and support each other?

Source

What credentials, independence factors, and track record may affect report credibility?
v1.0

SOC 2 Reliability Rubric

A practical framework that helps GRC and TPRM practitioners assess how much weight to give a SOC 2 report when making vendor trust decisions. The rubric provides standardized signals to identify reports that demonstrate audit rigor versus those that warrant additional scrutiny.

What this evaluates

Report reliability as evidence — not whether a vendor's controls meet your specific needs. This rubric helps you assess the quality of the audit work itself.

The Problem

SOC 2 reports vary widely in quality, but practitioners lack a shared way to assess that variability. Without standardized criteria, teams either treat all reports as equally credible, apply inconsistent subjective judgments, or waste time investigating every report from scratch. The result is unnecessary uncertainty for practitioners, inconsistent feedback for vendors, and an ecosystem that struggles to differentiate quality work.

Approach

The rubric evaluates reports across three dimensions — Structure, Substance, and Source. Structure failures indicate the report may not meet professional standards. Substance failures mean the documented work doesn't support the conclusions. Source failures suggest factors that undermine independence or credibility. Only by evaluating all three together can practitioners determine whether a report provides reliable assurance or merely creates the appearance of compliance.

Structure

Does the report include required components and maintain professional consistency?

Substance

Do the controls, testing, and conclusions logically align and support each other?

Source

What credentials, independence factors, and track record may affect report credibility?
Explore the Rubric

How to Use This Rubric

This rubric contains 11 signals across three pillars to evaluate SOC 2 report quality. Use the sidebar to jump to any pillar or signal, or scroll through the full rubric below.

Structure (3 signals)

Professional standards compliance

Substance (4 signals)

Audit work rigor and logic

Source (4 signals)

Auditor credibility factors

Structure

Does the report include required components and maintain professional consistency? Structure failures indicate the report may not meet professional standards.

S1

Required Auditor's Report Section Structure

Why It Matters

AICPA standards mandate specific paragraphs in the Auditor's Report: Scope, Opinion, and for Type 2, Description of Tests of Controls. Missing or incorrect paragraphs indicate the auditor is unaware of basic standard requirements or took shortcuts.

What To Look For

  1. Scan the Auditor's Report section (Section 1 or 2) for labeled paragraphs: Scope, Opinion, and Description of Tests
  2. For Type 2, verify there's a paragraph referencing tests in Section 4
  3. Check that the Opinion clearly states whether controls were suitably designed and operating effectively
  4. A qualified opinion will typically have an explanatory paragraph, then say "Except for the matters above…" — indicating the opinion has been qualified
  5. Ensure the opinions reflect the most recent format published by the AICPA
S2

Management's Assertion Completeness

Why It Matters

Management must formally assert their system description is accurate, controls are suitably designed, and (for Type 2) operating effectively. Missing or incomplete assertions mean management hasn't taken responsibility for their control environment per AICPA standards.

What To Look For

  1. Find Management's Assertion in Section 1 or as a separate section
  2. Verify it includes all required elements and is signed by company leadership
  3. If missing, incomplete, or unsigned, the report doesn't meet basic standards — request a complete version before proceeding with your assessment
S3

Inconsistent Language Across Report Sections

Why It Matters

Inconsistencies across report sections indicate copy-paste reuse, weak editorial control, or lack of holistic auditor review. These discrepancies tell us the audit firm either did not understand and evaluate the actual environment, or did not prioritize the report user's clarity of understanding when drafting the report.

What To Look For

  1. Compare systems described across Sections 1, 3, and 4 for alignment
  2. Watch for control frequencies that change between sections (e.g., "quarterly" in Section 3 but "annual" testing in Section 4)
  3. Look for different system names used to describe the same environment
  4. Check for services or activities that are kept out of scope but suddenly appear in other sections (e.g., org chart doesn't have a CISO but CISO is mentioned elsewhere)

Substance

Do the controls, testing, and conclusions logically align and support each other? Substance failures mean the documented work doesn't support the conclusions.

S4

System Description Specificity

Why It Matters

Section 3 should name actual products, technology stack components, infrastructure providers, and organizational structure. Generic buzzwords that could describe any company suggest the auditor didn't engage with the real environment.

What To Look For

  1. Look for specific details: AWS/Azure/GCP, named SaaS tools, data center locations, organizational charts, architecture diagrams, subservice organizations involved in providing services, and policies and procedures
  2. If it reads like marketing copy you could paste to any company, the auditor likely didn't test to the precision you'd expect
  3. Cross-reference against what you know about the vendor's actual tech stack (website details, questionnaire, subprocessor list, etc.) and other SOC 2 reports by the same auditor
  4. Ensure the System Description is consistent with the products or services described on the vendor's website or marketing materials
  5. Scrutinize descriptions with a very specific boundary that seems to exclude key pieces of the environment — e.g., a healthtech SaaS tool that excludes any details about third parties that process PHI
Pro Tip: The three dominant cloud providers all have Trust & Safety portals where clients with appropriate permissions can download officially signed compliance reports: AWS Artifact (https://aws.amazon.com/compliance/soc-faqs/), Azure Service Trust Portal (https://servicetrust.microsoft.com/), Google Cloud Trust Center (https://cloud.google.com/trust-center).
S5

Control-to-Criteria Mapping Logic

Why It Matters

Each control maps to Trust Services Criteria (like CC6.1 for logical access). Illogical mappings — like "annual meetings" mapped to technical access controls — suggest the auditor didn't think critically about what controls actually accomplish.

What To Look For

  1. Spot-check 10 control mappings and ask: does this control logically address this criterion?
  2. Flag cases where technical controls are mapped to wrong categories or soft controls are used for hard technical requirements
  3. Document questionable mappings and probe whether those areas are well-designed — poor mapping can mask control gaps and fail to ensure the audit properly tested an important control
S6

Vague or Conflicting Control Descriptions

Why It Matters

Vague controls like "Management maintains security" don't tell you what's actually happening. Clear controls specify what happens, who does it, how often, and what makes it effective. Controls that contradict each other indicate at least one is ineffective or inaccurately described.

What To Look For

  1. A well-designed control should answer five questions: What is done? How is it done? Who does it? When is it done? Where is it done?
  2. Look for controls requiring approvals that other controls explicitly bypass
  3. Watch for overlapping controls with different populations (e.g., "all users" vs "excluding service accounts" vs "no contractors")
  4. Check for contradictions — e.g., one section says developers have no production data access, another mentions limited production data access
  5. Review controls side by side in Section 3 and trace testing to Section 4

Good example: "Security team reviews production access quarterly, validates business justification with managers, removes unjustified access within 24 hours."

Bad example: "Access is reviewed periodically."

S7

Test Procedure Detail and Specificity

Why It Matters

Vague test descriptions like "reviewed evidence" or "inspected evidence" are unhelpful. Look for testing descriptions that indicate the test itself was reperformed or observed. Also ensure sample sizes are large enough to provide meaningful confidence. An unqualified report despite extensive exceptions can indicate conflict avoidance or pressure to preserve the client relationship.

What To Look For

  1. Pick 5–7 controls critical to your use case and read their test procedures line by line
  2. Look for: what evidence was examined, how many samples, from what time periods, what specifically was verified
  3. If procedures are interchangeable boilerplate that could apply at any company, flag these controls and request direct evidence from the vendor
  4. Samples should be selected from multiple dates during the monitoring period to ensure continuous assurance
  5. For technical controls (MFA, encryption), look for testing of configuration and system-generated evidence
  6. For periodic controls (quarterly reviews), verify all instances were tested
  7. Count exceptions across Section 4 and assess whether exceptions are pervasive and/or impact core security objectives — challenge whether the opinion appropriately reflects the severity
  8. Scrutinize non-occurring controls — ensure the audit firm disclosed meaningful validation to justify the non-occurrence

Source

What credentials, independence factors, and track record may affect report credibility? Source failures suggest factors that undermine independence or credibility.

S8

CPA Firm Registration, Peer Review Enrollment & Results

Why It Matters

The audit firm must be registered as a CPA firm with its respective State Board. The firm must be enrolled in the AICPA Peer Review Program and pass peer review every three years. Failures here indicate the audit may not be subject to proper oversight, and the final report may not meet AICPA quality standards.

What To Look For

  1. Find the firm name, firm signature, and home state at the bottom of Section 1 or Section 2
  2. Verify registration at NASBA's CPAVerify tool (https://ald.nasba.org/search/cpa) — the official nationwide firm lookup service
  3. If you can't confirm the firm is licensed, reject the report and communicate to the vendor that you cannot accept a SOC 2 from an unlicensed CPA firm
  4. Search the AICPA Peer Review public file (https://peerreview.aicpa.org) to validate enrollment — if the firm doesn't appear, they are not enrolled; reject the report
  5. If enrolled, ensure the Report Rating is "Pass" and the acceptance date is not older than three years
  6. New CPA firms have 18 months after their first attestation report to complete their first peer review — if outside that window, follow up for assurance on timing
Note: NASBA does not currently cover these states — search their Boards of Accountancy directly: ND · NE · NY · PA · WV · WY
S9

CPA-to-SOC Reports Issued Ratio

Why It Matters

Auditing standards require a licensed CPA to sign off on every SOC report. A high ratio of reports per CPA suggests the firm may be operating as a "signature mill" without prioritizing quality.

What To Look For

  1. Look up the CPA firm on LinkedIn and confirm how many licensed CPAs work there
  2. Research the firm to estimate the number of SOC reports they issue per year
  3. If the ratio of licensed CPAs to SOC reports issued per year is greater than 50:1, this is a signal that quality may not be prioritized
  4. Factor this into your assessment and ask for supplemental evidence
S10

CPA Firm Leadership & Report Signer Experience

Why It Matters

CPA leadership determines whether a firm operates as an independent guardian of security or a high-volume audit mill. Their oversight dictates whether staff apply rigorous professional skepticism or simply follow automated checklists. Report quality rests on leadership's willingness to prioritize their professional license and technical accuracy over easy revenue.

What To Look For

  1. Research the firm's founders, managing partner, and report-signing CPAs on LinkedIn for sufficient SOC 2 audit experience
  2. Check for AICPA membership and recognition
  3. Look for certified professionals: CFE (Certified Fraud Examiner), CISSP (Certified Information Systems Security Professional), CISA (Certified Information Systems Auditor)
  4. Evaluate industry experience relevant to your vendor's sector (healthcare, financial services, SaaS, etc.)
  5. Assess process transparency — does the firm share clear timelines, document requests, and readiness assessments before fieldwork begins?
S11

Use of a GRC Tool

Why It Matters

Some GRC tools market "instant" SOC 2 compliance, promising audits in hours or days and guaranteeing a "pass." Such marketing often signals a commodity audit that prioritizes speed over substance.

What To Look For

  1. Check the vendor's Trust Center — in many instances, the vendor is using the GRC tool's Trust Center product, revealing which tool they use
  2. If the vendor doesn't have a Trust Center, inquire directly about which GRC tool they used
  3. Research the tool's website for marketing claims: "SOC 2 in days, hours, or weeks" signals rushed risk assessments
  4. Look for "Audit-Ready Guarantee" or "100% Success Rate" — this signals a check-the-box culture where the auditor acts as a rubber stamp
  5. Check if the tool has a list of "preferred auditors" — if the vendor's auditor is on that list, their independence may be weakened by a high-volume, automated business model
  6. Factor findings into your assessment and ask for supplemental evidence

Responding to Quality Issues

Eight practical approaches for addressing report quality concerns

1

Focus on Education, Not Accusations

Approach vendors with curiosity and clarity, not blame. Many well-meaning vendors operating strong security programs may not understand what a high-quality SOC 2 report means, or were guided into a low-rigor audit by cost or sales pressure. Use this as an opportunity to enhance the relationship and grow their understanding.

2

Communicate with the Vendor

Don't silently downgrade trust. If material concerns are identified, explain what you're seeing and why it matters. Clear, specific feedback helps vendors improve and strengthens trust across the ecosystem. Frame it as: "This year we can do a manual review since the report wasn't of acceptable quality — we'll need to see specific changes in the next report to accept it."

3

Involve Stakeholders Early

Business owners, risk owners, and technical stakeholders need to be involved from the start. These teams are most impacted by delayed approvals, compensating controls, and risk acceptance — and often have critical context that informs the final decision.

4

Apply a Risk-Based Lens

Not all vendors carry the same risk. Consider data sensitivity, access level, deployment model, and business criticality. A low-impact vendor may warrant lighter scrutiny than a mission-critical system.

5

Identify Practical Mitigations

If the report isn't sufficient, consider alternatives before rejection. Request supplemental evidence for key controls, limit production access or scope of deployment, or delay rollout until improvements are made. Even partial approval or constrained adoption can create strong incentives to improve audit quality in future cycles. This approach also reaffirms a GRC team as a business enabler that makes risk-based decisions rather than the Team of No.

6

Use Commercial and Contractual Levers

When a SOC 2 report cannot be reasonably relied upon, additional assurance work may be necessary. The cost of supplemental reviews, evidence requests, or ongoing monitoring can be addressed through contract terms, negotiations, or security addenda. This may include requiring a higher-quality auditor in future cycles, mandating specific controls or audit procedures, or pricing in the additional cost of oversight. Your ACV and TCV as a client will significantly impact your ability to negotiate favorable terms.

7

Engage the Auditor

The audit community gets very little feedback from report consumers. Constructive engagement with specific concerns can improve future audits — not just for one vendor, but for the broader ecosystem.

8

Document the Evaluation

Whether risk is mitigated, transferred, or accepted, document the rationale. This is ultimately a risk-based exercise, and documentation supports your own governance obligations and business needs.

Community Projects

Vote on and contribute to initiatives that improve the SOC 2 ecosystem

About the Guild

Mission

The SOC 2 Quality Guild is a practitioner-driven community creating standardized evaluation criteria for SOC 2 audit report quality. We exist because practitioners deserve shared, objective ways to assess the reliability of the trust signals they depend on every day.

Why This Matters

SOC 2 has become the most widely adopted security assurance framework for SaaS companies. But rapid growth in demand has created a quality gap — reports vary dramatically in rigor, and the ecosystem lacks standardized ways to tell the difference. TPRM teams are left making decisions based on vibes, anecdotal experience, or brand recognition rather than observable, verifiable signals. By giving practitioners the tools to consistently evaluate report quality, we can create market pressure that improves outcomes for everyone — vendors, auditors, and the organizations that rely on their work.

Approach

We focus on education, not accusations. Our frameworks evaluate the reliability of reports as evidence — not the trustworthiness of individual vendors or auditors. We provide repeatable, verifiable signals that any practitioner can apply, regardless of experience level.

Get Involved

The Guild is open to GRC practitioners, TPRM professionals, auditors, and anyone who cares about improving the quality of security assurance.

GitHub · Slack

License

This work is licensed under CC BY-SA 4.0. © 2026 SOC 2 Quality Guild.