Automation and Permission Ceiling

Calibrant automates controls when Microsoft provides a suitable read path and asks you to validate the rest. Some manual controls are organizational — whether you have defined a break-glass procedure is not a tenant setting. A smaller set is technical but would require write-capable access or an unsupported authentication path, so Calibrant leaves it manual.

This page exists because that distinction is invisible from the outside. A vendor with a higher automation percentage may simply be asking for more privilege than we are.

The trade

Calibrant reads your tenant through a relay running in your own Azure subscription, authenticating as a managed identity you control. That identity holds read-only Microsoft Graph application permissions and exactly one directory role: Global Reader, which is read-only. Every write-capable directory role Calibrant once requested has been removed.

One surface is not read-only, and we would rather say so than let the sentence above imply otherwise. Power Platform exposes its admin API to service principals through a management-application registration, and that registration is not scoped read-only: we measured an identity holding the registration and no directory role at all, and it could both read environments and issue an admin delete against one. Microsoft offers no read-only variant of it. Calibrant issues only GETs, from a fixed catalogue compiled into the relay, but the registration itself permits more.

That registration is optional and opt-in. The setup script does not create it unless you pass -EnablePowerPlatform; a normal run grants read-only permissions only and says so. If you enabled it and changed your mind, remove the registration afterwards. Step 5 of Relay Setup states this before the script runs, and Removing the Relay has the removal command. Either way the Power Platform checks report not evaluated — you lose one baseline control and four CISA SCuBA controls, and nothing else changes. Declining is a supported configuration, not a broken install.

Read-only is not the same as narrow

The default relay grant contains 18 tenant-wide Microsoft Graph application roles. They cannot change the tenant, but they run without a signed-in user and some expose more data than a single Calibrant control needs. Protect the relay VM and its Managed Identity as a privileged reader. Three of the roles used for expanded Healthcheck evidence make this distinction especially clear:

Data minimization limits what the compiled Calibrant collector returns; it does not shrink the Microsoft permission itself. Anyone able to replace or run code on the relay VM could use the Managed Identity up to its full granted read scope. The VM therefore has no public inbound path, accepts only catalogued operations from Calibrant, and requires customer approval for relay updates. See the exact permission list and purpose and relay security architecture.

After using the available read-only APIs, the remaining ceiling is deliberate and costs us coverage. Every additional percentage point of automation beyond it would require granting a role that can change your tenant, in order to read it. We think that is the wrong trade for a product whose entire job is to tell you your configuration is sound, so we take the lower number and tell you which checks it costs.

In August 2026 we removed five roles that earlier versions had requested. First Exchange Administrator, Teams Administrator and SharePoint Administrator, after measuring that Global Reader covered the same cmdlets. Exchange Administrator alone permits New-TransportRule and Add-MailboxPermission: the ability to reroute and read mail, held by a product that only ever needed to look.

Then Power Platform Administrator and Fabric Administrator, which we had kept because no read-only alternative was known to work — wording that, on inspection, meant untested rather than tested and failed. Measuring it settled the question: those two surfaces authorise through their own registration mechanisms, not through any directory role. A test identity holding the Power Platform management-application registration and Fabric API security group but neither administrator role reads Power Platform environments, DLP policies, flow runs and all 169 Fabric tenant settings successfully. An identity holding only Global Reader gets 403 from Power Platform and 401 from Fabric. So the roles were granting nothing we used, while carrying the ability to delete Power Platform environments, rewrite DLP policy, and take over Fabric workspaces.

Dropping them cost zero controls. Nothing in the automation figures below changed. What it did not do is make Power Platform read-only — see the registration caveat above; removing the role narrowed the blast radius (no Power Platform admin centre, no Dynamics 365 administration) without changing what the admin API itself allows.

What it costs, specifically

Four SharePoint controls in the Calibrant M365 Security Baseline (three of which also appear in CISA SCuBA) are validated manually for this reason:

Microsoft Graph does not expose these four. That is not a limitation we assumed — we checked Microsoft's published API schema for both the v1.0 and beta endpoints, where the SharePoint settings resource declares an identical 29-property set and none of the four appears in either. Searching the entire schema for them returns nothing.

The only programmatic source is Get-SPOTenant in the SharePoint PowerShell module, which has no managed-identity authentication path at all, and whose alternatives require the SharePoint Administrator role — full tenant SharePoint control, to read four settings. So Calibrant does not install that module, and these four controls carry step-by-step verification instructions in the assessment instead: where to click in the SharePoint admin center, a read-only PowerShell command, and what a compliant answer looks like.

If Microsoft adds these properties to Graph, they become automatic checks with no change in the permissions we ask for. Until then, this is an honest boundary rather than a gap we are hiding.

The separate OneDrive external-sharing slider has the same API limitation, but Calibrant can still evaluate the Healthcheck alignment control and the Security Checkup's two sharing controls without SharePoint Administrator. The relay collects the SharePoint tenant sharing level, then the unpublished result asks an owner or administrator to select the OneDrive level shown in SharePoint admin center. Calibrant combines those two pieces into the verdict and reuses the answer only while the SharePoint evidence and evaluation rules are unchanged.

The two optional integrations, and what each costs you

Power Platform and Fabric are the only surfaces that need anything beyond the read-only Graph roles and Global Reader. Both are optional and both are opt-in: Fabric needs you to click through the admin portal yourself, and Power Platform needs an explicit -EnablePowerPlatform switch on the setup script. Neither happens on a default run. Here is what enabling each one buys, and what it asks for in return.

Fabric is the larger prize and the cheaper one: twenty controls, and the access it grants cannot write. If you enable only one, enable that.

The Power Platform trade-off, stated plainly

Power Platform is the exception to everything else on this page. Microsoft exposes its admin API to service principals through a management-application registration, and that registration is not scoped read-only — there is no reader variant to ask for instead. We measured what it actually permits, using an identity holding the registration and no directory role at all:

So an identity with this registration could, in principle, delete a Power Platform environment or rewrite DLP policy. Calibrant does not: the relay executes only its built-in, versioned scan catalogue, which contains reads and nothing else, and a compromise of Calibrant's own database cannot feed it different scripts. But the registration itself permits more, and we would rather you weigh that than discover it.

Declining is a supported configuration. Skip the registration and the Power Platform checks report not evaluated. You lose those five controls. Nothing else changes, and no other category is affected. Declining is also the default: simply run the setup script without -EnablePowerPlatform. If it is already registered, the removal command takes it away without redeploying the relay.

How the counts break down

The great majority of the manual controls in every framework are organizational: documented procedures, periodic reviews, and decisions that live in a policy rather than a settings page. SCuBA's share is larger because it was written for federal agencies and contains correspondingly more process guidance, and the Essential Eight's is larger still because its requirements span the whole IT estate rather than the Microsoft tenant alone.

A control is never counted as automated unless the scan can actually produce a verdict for it. A control the scan can only ever report as "unknown" is counted as manual, which is why these numbers are lower than they were before August 2026 — the definition got stricter, not the coverage worse.

Related

• Relay VM Security — what the relay is, and the exact permissions it holds
• Data Handling — what a scan reads and what it never touches
• Security Baseline — frameworks, control statuses and scoring