Scanning

This guide covers how to start a scan, what happens during execution, how scores are calculated, and how to read and export your results.

Starting a scan

1. Navigate to Healthcheck in the dashboard
2. Ensure the prerequisites banner shows your relay is online
3. Review Tenant Context (recommended, but not required to start)
4. Click Start Full Scan

Scans are included in your subscription — run them as often as you need. Every run uses the full product scope, so user-selected categories cannot make one run look better than another. Each scan also freezes verified license applicability. Gaining or removing a license can legitimately change which controls apply and therefore change the denominator; Insights compares only eligible like-scope results and keeps that scope explainable rather than promising every score movement is a configuration change. If Calibrant needs administrator context after reviewing the Microsoft evidence, the scan pauses at Context needed and publishes no score. After the last answer is saved, it resumes against the same frozen evidence. On later scans, matching saved answers are applied automatically; only context tied to changed evidence or evaluation logic is asked again. Technical collection gaps can still produce a provisional result that is kept out of Insights and automatic comparisons.

Scans created before Calibrant enforced the full eight-category scope remain visible as legacy history. They are not used for Insights, trend baselines, automatic comparisons, or current printable reporting. The first verified full-scope result starts the comparable trend line.

Scheduled scans (Pro)

On the Pro plan (with an active or trialing subscription), Calibrant can run a full healthcheck scan plus a security scan for each active assessment automatically every Monday at 06:00 UTC. Enable it under Settings → Scheduled Scans. Each promised run is put in a durable queue. Temporary application failures retry automatically; an offline or outdated relay is retried on a longer interval, up to the queue's bounded attempt limit. The Settings page records the final outcome, so “started” is never presented as “completed.” Scheduled scans pause while a subscription is past due. A scheduled healthcheck can collect before stable Tenant Context is complete, but it pauses without a score if administrator context is required. An owner or administrator must finish that scan's inline requests or cancel it before a later scheduled Healthcheck can start. Review Tenant Context before Monday and keep relay permissions healthy if you want that week's scan to become a final comparison point.

What happens during a scan

1. Context snapshot — Calibrant freezes current answers when the scan starts; stable Tenant Context remains eligible for 90 days, and a prior scan follow-up is applied only if the new Microsoft evidence and Calibrant evaluation version match what it reviewed
2. Command dispatch — Calibrant sends the fixed five-script collection set that covers all eight result categories to your relay
3. Relay execution — the relay runs each script using its Managed Identity, collecting configuration data from Microsoft Graph, Exchange Online, Teams, SharePoint, and Power Platform APIs
4. Data return — raw results are returned to Calibrant and parsed into structured findings
5. Evaluation — each finding is evaluated against health rules and assigned a preliminary status (pass, fail, not licensed, not applicable, not evaluated, or unavailable)
6. Focused follow-ups — when Microsoft evidence cannot prove intent, scope, exclusions, policy quality, or an allowed outside safeguard, the scan pauses and asks an owner or administrator for that missing context
7. All-or-nothing save — findings, score, summaries, license evidence, questionnaire evidence, and the audit-method identity become one completed result only when the full expected rule set agrees; otherwise Calibrant publishes no trusted completed audit
8. Optional AI analysis — after an eligible result is finalized, Claude can generate an executive summary and prioritized remediation recommendations; provisional and incomplete-evidence results are excluded

Full scans generally complete in minutes, depending on tenant size and Microsoft API response times. You can watch progress in real time — the UI shows which workloads have finished collecting.

Understanding scores

Scores range from 0 to 100 and are calculated per-category and overall. The scoring system is severity-weighted — not all checks count equally. Critical and high-severity findings carry significantly more weight than low-severity issues, reflecting their real-world impact on your tenant's security and operational posture.

Calibrant does not publish a score while an answerable context request remains. A provisional historical or technically incomplete score covers only the checks that could be evaluated. It remains available in scan history for review and export, but is excluded from Insights, trends, baselines, and automatic scan-over-scan comparisons.

How weighting works

Every health rule has a severity level. When your score is calculated, each passing or failing check contributes points proportional to its severity. A single critical failure won't tank your entire score, but multiple high-severity failures will pull it down meaningfully. Low-severity items still matter, but they won't overshadow more important findings.

What's excluded from scoring

Your score only reflects checks that were actually evaluated. The following are excluded from both the numerator and denominator — they don't help or hurt your score:

• Not licensed — the feature requires a license tier your tenant doesn't have (e.g., Conditional Access requires Entra ID P1)
• Not applicable — a permitted, documented answer proves that the rule is outside scope, such as Viva Engage not being used or Windows provisioning genuinely being outside the tenant's scope
• Not evaluated — data couldn't be collected, usually due to a missing permission or transient API error
• Unavailable — a workload result or required inventory was not returned, so Calibrant does not guess
• Info-only checks — informational observations with no pass/fail criteria

This means features proven unlicensed do not help or hurt the score. A check that could not run is not guessed as a pass or fail, and its incomplete evidence keeps the result out of automatic trends and comparisons.

Score ranges

Reading results

After a scan completes, the results page shows:

• Context needed, final, provisional, or incomplete-evidence status — whether required administrator decisions and technical evidence are complete
• Scan coverage — how many checks were evaluated, passed, failed, and skipped
• Overall score — your aggregate score across all categories
• Category scores — individual scores for each applicable category (click a category to filter findings)
• AI executive summary — for eligible final scans, a generated analysis of your tenant's health posture
• Prioritized recommendations — for eligible final scans, AI-generated action items ranked by effort and impact

Finding statuses

Use the filter tabs to narrow findings by status (Failing, Passing, Not Evaluated, Not Licensed, or Not Applicable) or by category. Expand any finding to see detailed information and remediation guidance.

Exporting results

Two export formats are available from the results page:

• CSV — spreadsheet-friendly export of all findings with rule ID, category, severity, status, message, remediation, result completeness, and health audit coverage
• Raw JSON — owner/admin-only operational evidence containing the full relay output for each category, useful for auditing or integration with other tools

Troubleshooting

• Scan shows “Context needed” — Microsoft collection finished, but one or more controls still need an owner or administrator to confirm scope, intent, exclusions, or an allowed alternative. Open the scan, review what Microsoft showed, and save each requested answer. The same scan resumes after the last answer.
• Relay shows offline on the Healthcheck page — go to Connections and check that your relay deployment has a green Online indicator. If it's offline, view the relay logs via Azure Run Command: az vm run-command invoke --name calibrant-relay-vm --resource-group calibrant-relay-rg --command-id RunPowerShellScript --scripts "Get-Content C:\calibrant-relay\relay.log -Tail 30"
• Scan fails with “Access Denied” — the Managed Identity is missing one or more permissions. Re-run the permission grant script and wait 5–10 minutes for propagation.
• Exchange or Teams category fails but others pass — the Global Reader role may not have propagated yet, or (on relays granted before August 2026 and not re-granted since) the legacy Exchange/Teams Administrator roles were removed without re-running the grant script. Wait 10 minutes and retry, then re-run the grant script.
• Power Platform shows no data — the relay's Managed Identity must be registered as a Power Platform management application. See Step 6 of the Relay Setup guide for the BAP API registration step. The integration is optional; without it, Power Platform findings show as not evaluated and do not affect other category scores.
• Scan stuck in “Collecting” — Calibrant accepts slow relay results while the full workload set is still arriving, but never publishes a partial score. If collection cannot finish within 90 minutes, the scan is marked failed automatically. Check relay logs and restart the service if needed: az vm run-command invoke --name calibrant-relay-vm --resource-group calibrant-relay-rg --command-id RunPowerShellScript --scripts "Restart-Service CalibrantRelay"