Skip to main content

​
After this article

You will be able to connect Databricks to Lumos to sync and provision users, service accounts, groups, account roles, repository permissions, and workspace assignments across your Databricks account.

​
Capabilities

​
Required plan and roles

  • Databricks role: an account admin is required to complete the setup. The service principal Lumos uses must itself be granted the Account admin role.
  • Workspace access: for Lumos to manage workspace-level data (workspace groups, repo permissions, workspace assignments), the service principal must additionally be granted Admin permission on each workspace.
  • Credentials: an OAuth service principal (Client ID + Secret) created in the Databricks account console. All connector operations use the all-apis OAuth scope.
  • Permission scoping: this integration supports both syncing and provisioning and can only be connected with read + write access. There is no read-only variant. Account API operations require account administrator permissions. Workspace API operations require workspace administrator permissions.

​
Read-only access vs provisioning

Is read-only possible? No. Databricks authenticates with a single OAuth service principal using the all-apis scope, and that same scope backs every read and write operation. No narrower, read-only credential is documented. What full provisioning requires. An OAuth service principal created in the account-level UI, granted the Account admin role, plus Admin permission granted manually on every workspace Lumos should manage (workspace access is not inherited from the account role). Follow the steps below. Watch out for. Workspace access has to be granted per workspace. Having Account admin on the service principal is not enough on its own. Account-level operations succeed but workspace-level ones (workspace groups, repo permissions, workspace assignments) fail until you add the service principal with Admin permission under that workspace’s Permissions tab.

​
Instructions

Phase 1: Create a service principal in Databricks
  1. In the account-level UI, go to User management → Service principals tab → Add service principal.
  2. Enter a name, click Create new service principal, then Add service principal.
  3. Open the new service principal, go to the Roles tab, and check Account admin.
  4. Go to Credentials & secrets. Under OAuth secrets click Generate secret and enter a lifetime. Note the Client ID and Secret.
  5. Grant the service principal access to each workspace (required, workspace access is not inherited). Go to Workspaces, click the workspace, open the Permissions tab, click Add permissions, type the service principal name, select Admin permission, and click Save. Repeat for every workspace Lumos should manage.
  6. Retrieve your Account ID from the account console (AWS/GCP/Azure) via the down arrow next to your username.
  7. Determine your Instance Type from the account console URL: accounts.cloud.databricks.com = AWS, accounts.gcp.databricks.com = GCP, accounts.azuredatabricks.net = AZURE.
Phase 2: Connect in Lumos
  1. Go to Settings → Integrations → Databricks (reconnect an existing card or add a new one).
  2. Enter the Client ID, Client Secret, Account ID, and Instance Type (AWS, AZURE, or GCP).
  3. Click Connect.

​
API Endpoints Used

​
Troubleshooting

Workspace data missing or workspace operations fail. Workspace access must be granted manually per workspace. Account admin on the service principal is not enough. Add the service principal with Admin permission under the workspace’s Permissions tab. Sync log shows “Permission assignment APIs are not available for this workspace”. Some workspaces don’t support the permission-assignment APIs and return 400 for those calls. Lumos skips this error so the full sync still completes for those workspaces. No action is required. Connection fails with valid credentials. Verify the Instance Type matches your account console URL (accounts.cloud.databricks.com = AWS, accounts.gcp.databricks.com = GCP, accounts.azuredatabricks.net = AZURE) and that the Account ID was copied from the account console. OAuth secret expired. Service principal OAuth secrets are created with a lifetime. Generate a new secret under Credentials & secrets and reconnect. New users created without expected entitlements. Effective September 14, 2026, Databricks no longer grants new users default workspace entitlements automatically. Lumos provisioning explicitly assigns the chosen entitlements/access level at account creation, so select the intended access level when granting.