Azure Databricks cluster nodes should have no public IP addresses

Without secure cluster connectivity, the virtual machines behind your clusters are given public IP addresses, which makes them reachable from the internet if a network rule is too loose. With it, the customer virtual network has no open ports and the cluster nodes have no public IP addresses.

Instead, each cluster opens an outbound connection on port 443 to a relay in the Databricks control plane when it is created, and administration requests travel back through that tunnel. Microsoft Learn recommends turning it on for the most secure deployment and says Azure Databricks plans to make it mandatory for classic workspaces in a future release.

Source: Enable secure cluster connectivity, Microsoft Learn.

In the Azure portal, open the workspace, then Settings and Networking. On the Network access tab, Deploy Azure Databricks workspace with Secure Cluster Connectivity (No Public IP) should read Enabled.

The same read-only check from the Azure CLI:

az databricks workspace show \
  --resource-group <resource-group> \
  --name <workspace-name> \
  --query "parameters.enableNoPublicIp.value"

true is the safe answer. Microsoft Learn also describes a physical check: start a cluster, open a VM in the managed resource group and confirm its public IP address field is empty.

New workspaces created in the Azure portal or from an ARM template of recent versions have this on by default. For an existing workspace, it must already use VNet injection. Stop all classic compute first (clusters, pools, classic SQL warehouses), then in the portal go to Settings, Networking, Network access, set the option to Enabled and select Save. Microsoft Learn says the update can take over 15 minutes.

A scripted option (run it yourself; ConfigCobra never changes your tenant):

az databricks workspace update \
  --resource-group <resource-group> \
  --name <workspace-name> \
  --enable-no-public-ip true

Terraform users on azurerm 3.41.0 or later can change no_public_ip from false to true without recreating the workspace.

Data engineers and analysts see compute stopped during the update, so schedule it for a quiet window. Afterwards the workspace subnets are private, so outbound internet access needs an explicit route. With the default managed network Azure Databricks creates a NAT gateway, which carries an extra cost; with VNet injection the network team should provide one, ideally with a stable outbound IP for external allow lists.

If a firewall or network security group controls the cluster subnets, its rules may need updating: there is one additional outgoing connection to the control plane, and incoming connections from it are no longer used. Allow the relay by domain name rather than by IP address.