Security Baseline

Security Baseline is a framework-driven compliance assessment for your Microsoft 365 tenant. Unlike a one-time scan, it maintains a living assessment — the relay auto-detects what its read-only collectors can prove, and you manually validate registered manual controls. Where an exact control explicitly permits an equivalent third-party safeguard, you can submit evidence for compensated credit. The result is a living assessment view that combines the latest eligible scan evidence with the attestations your team maintains.

How it works

1. Choose one or more frameworks (Calibrant M365 Security Baseline, CISA SCuBA M365 Baselines)
2. Calibrant creates your assessment with all framework controls listed as Pending
3. Optionally record integration intent for Fabric/Power BI and Power Platform
4. Run a scan — the relay collects M365 configuration data via Managed Identity
5. If Microsoft omits an exact detail needed for a verdict, answer only the focused context shown on that scan; no score is published while an answer is outstanding
6. Controls with M365-automatable checks are updated to Auto Pass or Auto Fail
7. Complete the stated validation steps for manual controls
8. For an explicitly eligible control handled by another tool, document the control-specific safeguard, tool, evidence reference, and reviewer
9. Your compliance score updates as you complete attestations and re-scan

Prerequisites

1. Microsoft commercial cloud tenant — Microsoft 365 GCC, GCC High, DoD, China/21Vianet, and every other sovereign or national cloud are not supported.
2. Relay deployed and online — follow the Relay Setup guide
3. Managed Identity permissions granted — same permissions as Tenant Healthcheck. Follow Step 5 of the Relay Setup guide.
4. Active access — Security Baseline is included in the 14-day trial and in both Essential and Pro.
5. Optional integration intent — record whether Fabric/Power BI and Power Platform should be collected automatically or reviewed manually. Successful automatic evidence needs no questionnaire confirmation. See the Prerequisite Questionnaire guide.

How licensing is detected

Calibrant does not decide eligibility from product names such as E3, E5, Business Premium, or E7. Every full scan reads the tenant's active Microsoft subscriptions and matches the underlying Microsoft service-plan identifiers to a versioned commercial-cloud registry. This means E3 plus a qualifying add-on, or a newer bundle such as E7, works when it contains the same required service plan; the bundle name itself does not need to be added to a list.

A detected capability means Calibrant may attempt the related tenant check. It does not certify that every benefiting user has the correct seat assignment, and Security Baseline is not a Microsoft licensing-compliance audit. That distinction matters for tenant-wide features that can become available after only one qualifying license is purchased.

• Native required — missing capability or a failed native setting remains not met; a note cannot turn it into a pass.
• Native or compensating — the exact control may receive time-limited compensated credit for an evidenced third-party safeguard. The native result remains separate and visible.
• Optional scope — N/A is available only after a complete scan positively proves the registered capability is absent.

Unknown plans, incomplete license data, and unsupported cloud environments fail closed. They are never guessed to mean "not licensed." A scan can be started before this inventory is known, but Calibrant will not publish a final assessment while a required license decision remains unresolved.

A scan that discovers a missing native prerequisite may finish as failed, but its verified license inventory can still support a control-specific third-party attestation for the same assessment. That inventory must be no more than seven days old when the decision is recorded. The accepted safeguard then has its own 90-day review period. Rerun the full scan after recording the safeguard so Calibrant can publish a final assessment that includes it.

Evidence coverage

A successful database save does not by itself make a scan a complete audit. Security Checkup labels each scan using the evidence that was actually available. Where Microsoft returns only part of a verdict, the result page pauses and asks an owner or administrator for the exact missing context before publishing:

• Complete — every integration-dependent automatic check had definitive evidence and any requested result-driven context, or Fabric/Power BI was marked Not in use with a saved explanation and evidence reference. This result can update the living assessment and participate in comparisons.
• Provisional — automatic evidence was unavailable for one or more integration-dependent checks, and the frozen choice said a person would review them manually. The findings remain useful, but this is not a final or comparable audit.
• Failed — a technical collection or integrity requirement failed, or the tenant's cloud/license evidence was unsupported or could not be verified. An answerable context gap pauses instead of publishing a failed partial result. A failed scan ends as Failed, so no completed result, score, assessment update, comparison, or printable report is produced.

Result-driven context currently covers the separate OneDrive sharing slider that Microsoft Graph omits and Fabric settings that Microsoft reports as enabled without their security group scope. Calibrant does not request SharePoint Administrator for the OneDrive slider. Answers are reused only while the relevant Microsoft evidence and evaluation rules are unchanged, so a later scan asks again only when that control's basis changed.

This integration coverage is separate from ordinary manual framework controls. A normal manual assessment control can remain Pending and still need an attestation without making the scan's Fabric/Power BI or Power Platform integration coverage provisional.

Licensing decisions are also reported separately from collection coverage. If Microsoft evidence is complete but the linked framework requires capabilities the tenant does not have, the result is labelled Framework license decision required. Calibrant lists the exact affected controls and says whether each requires the native Microsoft capability or permits an evidenced compensating safeguard. The scan evidence is retained, but the living assessment is not updated until those decisions are resolved and a new scan runs. This state does not mean the relay failed to collect an automatic check.

Use the scan result's resolution link to open a focused workflow containing only those controls. For each one, either add the missing Microsoft capability or document a genuinely equivalent third-party safeguard when that exact control permits compensation. If neither is in place, leave it unresolved—the framework assessment is not eligible for final publication. Then run a new scan; Calibrant retains the blocked scan as history instead of rewriting it.

Supported frameworks

Calibrant M365 Security Baseline v1.0

Calibrant's independently authored baseline — every control maps to a specific M365 configuration requirement. 140 controls across 9 areas (Tenant Administration, Email & Threat Protection, Data Protection, Device Management, Identity & Access, Exchange, SharePoint & OneDrive, Teams, Fabric & Power BI). Controls are rated Level 1 (minimum recommended) or Level 2 (high-security environments). You can target L1 or L2 from the assessment settings.

94 of 140 controls currently have an automatic check; the other 46 are validated manually. Many are organizational rather than technical (for example, documenting an emergency-access procedure, an incident-response plan, or separation of duties) — there is no tenant setting that proves the process. PIM, recurring access-review, and Purview coverage now use automatic or hybrid relay evidence when licensed. A smaller number are technical settings that Calibrant could read only by holding administrative permissions it deliberately does not ask for; those carry step-by-step verification instructions in the assessment. See why some checks are manual. Controls may carry informational cross-references to CIS Microsoft 365 Foundations Benchmark v6 and CISA SCuBA where Calibrant's authors identified alignment. The current CIS references are a v6 mapping aid; they do not make Calibrant's independently authored baseline an official CIS Benchmark assessment, a claim of v7 coverage, or a certification.

CISA SCuBA M365 Security Baselines

CISA's Secure Cloud Business Applications (SCuBA) project provides security configuration baselines for Microsoft 365. Calibrant currently bundles a November 19, 2024 snapshot containing 134 controls across M365 products including Entra ID, Defender, Exchange Online, Power Platform, SharePoint Online, OneDrive, Teams, and more.

68 of 134 controls currently have an automatic check; the other 66 are validated manually. The proportion is lower than the baseline's because SCuBA includes a good deal of policy and process guidance (agency-level decisions, documented procedures) that no scan can answer. Each SCuBA control also shows its cross-mapping to the relevant baseline control.

Control statuses

Compliance score

The score formula is:

Score = (Auto Pass + Attested) / (Total − N/A) × 100

Pending and not-met controls count against the score. A user cannot improve the score by choosing N/A for a universal obligation; exclusions are limited to registered cases with supporting evidence. Per-section scores are calculated the same way for each framework section or SCuBA product baseline.

Security trends never combine unlike evidence: points are separated by the exact assessment, framework, and L1/L2 target that produced them. Changing an assessment from L1 to L2 does not relabel its older scores, and two assessments are not presented as one timeline.

Manual attestation

When a control is not auto-detectable, or a registered policy permits a third-party safeguard, click the control to expand it and follow the displayed evidence contract:

• Manual validation — for a deliberately manual control, complete its read-only validation steps and record the required note and evidence reference.
• Compensating control — only for a control explicitly marked eligible, identify the equivalent safeguard, tool, and evidence reference. The record is bound to that exact control and policy, expires after 90 days, and is shown as compensated rather than as a native Microsoft pass.
• Not Met — acknowledge the gap. Useful for tracking known issues you plan to remediate.
• N/A — available only where the registered control policy permits a scope exclusion and Calibrant has the required verified license or not-in-use evidence.

A control-specific compensating safeguard may be recorded whether the Microsoft capability is absent or the native setting failed. The failed native result remains visible. One attestation never satisfies other controls merely because they use the same product or tool. Calibrant requires recent verified license evidence from that same assessment before it accepts a service-plan-based exception.

Scan cost

Scans are included in your subscription and don't consume credits — run them on demand as often as you need. Creating and updating an assessment (including manual attestations) is likewise included.

Exporting results

From the assessment detail page, use the Export CSV button to download all controls with their current status, evidence/notes, and tool names. Columns include: Framework, Section, Control ID, Title, Level, Status, Evidence/Note, Tool, Result completeness, and Security evidence coverage. Scan-result CSVs also include the frozen prerequisite evidence for owners and administrators; members never receive those sensitive answers or references.

The printable report shows the assessment state that Calibrant can verify under the current report and framework contract. It refuses a verified presentation when the control set, target level, score, source scan, or audit method does not agree. Calibrant retains the old scan's score and compact audit context, but does not promise that a future product version can reproduce the exact visual appearance of a PDF that was not saved at the time.

Download PDF (available on every plan, for the assessment report and the Healthcheck scan report) renders that same printable report to an audit-ready PDF on Calibrant's own servers — the document is generated from the same frozen evidence as the on-screen page, no third-party rendering service is involved, and nothing leaves the tenant boundary except your download. If the report would refuse to render on screen, the PDF export refuses with the same reason instead of producing an incomplete document.

Where to export

1. Open Security and select the assessment. Scroll past the score summary to the action bar — the row that reads Export CSV · Report · Download PDF · Run Scan. Download PDF renders the document (a few seconds) and saves it directly.
2. Or click Report first to open the printable report in a new tab and review it on screen — the same Download PDF button floats at the top right beside Print / Save as PDF.

Report and Download PDF appear for owners and administrators (the document embeds owner/admin-only prerequisite evidence), and the action bar is set aside while an assessment is paused for licensing decisions.

Reports and PDFs are generated only against the current ruleset. When Calibrant's checks for a framework have been updated since an assessment's last scan, the report names that as the reason and declines until a new scan runs — a document produced from outdated rules could claim verification the current rules might not reproduce. Older scans stay available in Scan History for reference, but they are never re-presented as a current verified report.

Troubleshooting

• Entra ID controls show "Unavailable" — the relay may not have the Graph permissions for Conditional Access and Identity Protection. Re-run the permission grant script and ensure Policy.Read.All and IdentityRiskyUser.Read.All are included.
• Power Platform DLP shows an error — the Managed Identity must be registered as a Power Platform management application via the grant permissions script. If you intentionally omit that access, choose Manual review and record where the review will be verified. Any dependent gaps will then be labelled provisional rather than being mistaken for a complete audit.
• Scan stuck in "Collecting" — same as Tenant Healthcheck. Check relay logs and restart if needed.
• Score didn't update after attestation — the score recalculates immediately on save. Refresh the page if the number appears stale.