Home / Databricks on Azure

Databricks on AzureRunbookBeginner

Databricks App deployment failed or app crashing on Azure: logs, port, app.yaml and service principal checks

Work through a failed or crashing Databricks App in order - build and app logs, port binding, app.yaml, dependencies and the app service principal's access to code and data.

Deepak Bhardwaj7 min read

Applies toDatabricks Apps on Azure Databricks (Premium tier), Databricks CLI with the apps command group

Symptom

One of these, usually right after a deploy:

  • The deployment shows as failed and the previous version is still running, or nothing is.
  • The deployment succeeds but the app URL returns an error or never loads.
  • The app loads, then fails on its first query with a permission error.
  • The app worked yesterday and is now stopped or crashing with no code change.

Work through the steps in order. Most failures are found in the first two.

Step 1: Get the state and the logs

APP=my-app

databricks apps get "$APP" -o json | \
  jq '{url, app_status, compute_status, active_deployment, pending_deployment, service_principal_client_id}'

databricks apps list-deployments "$APP"

databricks apps logs "$APP" --tail-lines 500

Names in the output can vary by CLI version. If a field comes back null, look at the full JSON.

To watch a deploy live:

databricks apps logs "$APP" --follow --timeout 10m
databricks apps logs "$APP" --source SYSTEM --search error

You can also append /logz to the app URL in a browser, or open the logs in the app's page in the workspace. The log sources tell you which stage failed:

Log source What it covers Look here when
Build Dependency install and build scripts The deployment fails
System Platform events for the app The app is stopped or restarting
App Your code's stdout and stderr The app starts, then crashes or errors
HTTP Requests in Apache Combined Log Format Users report errors but the app log is quiet

Warning

Logs aren't kept when the app compute shuts down. Save them before you stop or redeploy: databricks apps logs "$APP" --tail-lines 0 --output-file app.log. Note that --output-file writes in addition to stdout.

The App log only contains what your code writes to stdout and stderr. If your app logs to a local file, the log will look empty. Fix that first.

Step 2: Match the symptom

Symptom Likely cause Fix
ModuleNotFoundError in the App log Package missing from requirements.txt Add it and redeploy
pip install fails in the Build log Version conflict, or egress to the package index blocked Pin compatible versions, or check workspace network controls
Build ignores package.json scripts No build script defined, or wrong package manager lock file Define build, and commit pnpm-lock.yaml if you use pnpm
App never becomes reachable Listening on localhost or a hard-coded port Bind 0.0.0.0 on DATABRICKS_APP_PORT
Wrong file runs No command in app.yaml, so the default was used Set command explicitly
Env var is literally empty or unset Variable defined outside app.yaml, or valueFrom key doesn't match Define it under env, and match the resource key
Deploy can't read source App service principal has no access to the folder Grant access to the source folder
Query fails with permission denied Service principal lacks grants, or the user token lacks a scope See step 5
App killed under load Memory exhausted on the current size Reduce usage or move to a larger compute size

Step 3: Fix the port and the start command

Your app must listen on 0.0.0.0 and on the port in DATABRICKS_APP_PORT. Requests arrive through a reverse proxy that handles TLS, and the app must support HTTP/2 cleartext (H2C).

The platform sets host and port variables for common frameworks, such as Streamlit, Gradio, FastAPI, Uvicorn and Flask. Hard-coded values in your code or start command override that help, so remove them.

# app.yaml
command: ['gunicorn', 'app:app', '-w', '4', '--bind', '0.0.0.0:$DATABRICKS_APP_PORT']
env:
  - name: 'DATABRICKS_WAREHOUSE_ID'
    valueFrom: 'sql-warehouse'
  - name: 'LOG_LEVEL'
    value: 'INFO'

Three rules catch most app.yaml mistakes:

  • The command doesn't run in a shell. Pipes, && and $OTHER_VAR won't work. DATABRICKS_APP_PORT is the one variable substituted in the command.
  • Variables defined outside the app configuration aren't available to the app. Declare them under env.
  • valueFrom takes the resource key from the app's resource list, not the resource's ID or name.

Step 4: Fix dependencies

The deployment runs these steps in order:

  1. If package.json exists, it installs Node packages: pnpm install --frozen-lockfile when pnpm-lock.yaml exists, otherwise npm install.
  2. If requirements.txt exists, it runs pip install -r requirements.txt. If not, and both pyproject.toml and uv.lock exist, it runs uv sync.
  3. If package.json defines a build script, it runs that.
  4. It starts the app with the app.yaml command, or the default command if there's none.

pnpm apps must set a command in app.yaml. Pin versions in requirements.txt so that a new upstream release can't break a redeploy of unchanged code. The Environment tab of the app shows what's preinstalled, so check it when a package you didn't list behaves oddly.

Step 5: Fix identity and data access

Each app has its own service principal. It uses DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET, which are injected automatically. Don't hard-code a personal access token as a workaround.

Service principal access

Check three things:

  • It can read the source folder in the workspace.
  • It has the right permission on each resource: for example CAN USE on a SQL warehouse or CAN QUERY on a serving endpoint.
  • It has Unity Catalog grants on the data.

Adding a resource needs Can manage on both the resource and the app. Attach resources under App resources, which also creates the key you reference in valueFrom.

-- Use service_principal_client_id from "databricks apps get"
GRANT USE CATALOG ON CATALOG sales TO `00000000-0000-0000-0000-000000000000`;
GRANT USE SCHEMA ON SCHEMA sales.reporting TO `00000000-0000-0000-0000-000000000000`;
GRANT SELECT ON TABLE sales.reporting.daily_orders TO `00000000-0000-0000-0000-000000000000`;

User authorization

If the app acts as the signed-in user, the token arrives in the x-forwarded-access-token header:

import streamlit as st

token = st.context.headers.get("x-forwarded-access-token")

The token only works in the app's workspace, and only within the scopes configured for the app. The default scopes (iam.access-control:read and iam.current-user:read) don't give access to data or compute. If only user-mode calls fail, add the scope you need, such as sql, and check that the user holds the Unity Catalog grants as well.

Who can open the app

This is separate from data access. CAN USE lets someone use the app, and CAN MANAGE lets them change it. Anonymous access isn't supported.

databricks apps get-permissions "$APP"

Step 6: Roll back or restart

There is no rollback button. Redeploy a known good version.

# Git-backed app: redeploy an earlier commit or tag
databricks apps deploy "$APP" --git-commit <old-commit-sha>
databricks apps deploy "$APP" --git-tag v1.0.0

# Workspace-backed app: deploy a folder holding the last good code
databricks apps deploy "$APP" --source-code-path /Workspace/Shared/apps/my-app-v1

# Restart the last active deployment
databricks apps stop "$APP"
databricks apps start "$APP"

Caution

Changing the Git repository, or switching between Git and workspace sources, deletes all Git credentials for the app's service principal. Recreate them with databricks git-credentials create before the next Git deploy.

Prevent it next time

  • Log to stdout and stderr, and write anything you need to keep to a Unity Catalog volume or table.
  • Deploy from Git tags or commits, so that rollback is one command. Admins can enforce this with Only allow app deployments from Git, under Settings > Development > Apps.
  • Pin dependencies and commit your lock files.
  • Handle SIGTERM. Your app must exit within 15 seconds or it receives SIGKILL, which can cut off in-flight work.
  • Size deliberately. The default Medium size has up to 2 vCPUs and 6 GB. Check the current options and costs before you resize.