Tenant Healthcheck
The Tenant Healthcheck product scans your Microsoft 365 tenant configuration against industry best practices and operational standards. It runs entirely through your Calibrant Relay using Managed Identity — no credentials leave your environment.
How it works
- You click Start Scan and select which categories to check
- Calibrant sends PowerShell scripts to your relay as commands
- The relay executes each script using its Managed Identity (no user credentials)
- Results are returned to Calibrant, scored against 56 health rules
- Claude AI generates an executive summary and prioritized recommendations
No M365 OAuth consent required. Unlike Agent Optimizer (which uses your M365 OAuth connection for persona authentication), Healthcheck uses only the relay's Managed Identity. Your Calibrant M365 connection is not needed for this product.
Prerequisites
- Relay deployed and online — follow the Relay Setup guide
- Managed Identity permissions granted — the relay's Managed Identity needs Graph app roles and M365 admin roles. Follow Step 4 of the Relay Setup guide to run the permission grant script.
- Professional or Enterprise plan — Healthcheck is a second product slot. Free and Starter plans include one product slot (Agent Optimizer by default). Upgrade your plan and select Tenant Healthcheck from your product selection.
Check categories
You can select any combination of categories when starting a scan:
| Category | Checks | Requires |
|---|---|---|
| Tenant | Org branding, technical contacts, directory sync health, service health incidents, delegated admin relationships | Graph app roles |
| Entra ID | Global admin count, guest settings, app registrations, password policy, privileged roles, stale guests, consent policies | Graph app roles |
| Exchange Online | Modern auth, audit logging, DKIM, IMAP/POP, transport rules, TLS connectors, distribution groups, SPF records | Exchange.ManageAsApp + Exchange Administrator role |
| Microsoft Teams | Upgrade mode, guest access, external federation, consumer access, team ownership, app policies, meeting settings | Graph app roles + Teams Administrator role |
| SharePoint Online | External sharing level, domain restrictions, resharing, legacy auth, anonymous link expiry, default link type, idle sign-out | SharePointTenantSettings.Read.All + SharePoint Administrator role |
| OneDrive for Business | Storage quota, sync restrictions, orphaned account retention, sharing alignment with SharePoint | SharePoint Administrator role |
| Power Platform | DLP policies, environment inventory, flow failures (last 7 days) | M365 CLI via Managed Identity (optional) |
Understanding scores
Each category gets an independent score (0–100), and an overall score is calculated from all findings weighted by severity:
| Severity | Weight | Examples |
|---|---|---|
| Critical | 20 pts | Modern auth disabled, no DKIM |
| High | 10 pts | Global admin count wrong, anonymous sharing open |
| Medium | 5 pts | Guest invite restrictions not set, IMAP enabled |
| Low | 2 pts | No branding, empty distribution groups |
| Info / Good | 0 pts | Informational observations, passed checks |
| Score range | Label |
|---|---|
| 90–100 | Excellent |
| 80–89 | Good |
| 60–79 | Needs Attention |
| 40–59 | Poor |
| 0–39 | Critical |
Credit cost
Each scan costs 2 credits regardless of how many categories are selected. Credits are deducted when the scan starts.
Troubleshooting
- Relay shows offline on the Healthcheck page — go to Connections and check that your relay deployment has a green Online indicator. If it's offline, view the relay logs via Azure Run Command:
az vm run-command invoke --name calibrant-relay-vm --resource-group calibrant-relay-rg --command-id RunPowerShellScript --scripts "Get-Content C:\calibrant-relay\relay.log -Tail 30" - Scan fails with "Access Denied" — the Managed Identity is missing one or more permissions. Re-run the permission grant script and wait 5–10 minutes for propagation.
- Exchange or Teams category fails but others pass — the directory roles (Exchange Administrator, Teams Administrator) may not have propagated yet. Wait 10 minutes and retry.
- Power Platform shows no data — the M365 CLI login with Managed Identity requires additional Power Platform API permissions. This category is optional; skipping it does not affect other category scores.
- Scan stuck in "Collecting" — relay commands expire after 10 minutes. If the relay goes offline mid-scan, the scan will be marked failed automatically. Check relay logs and restart the service if needed:
az vm run-command invoke --name calibrant-relay-vm --resource-group calibrant-relay-rg --command-id RunPowerShellScript --scripts "Restart-Service CalirantRelay"