Databricks Repos for Real Work and Certification Prep

Developing notebooks in Databricks is convenient because everything happens in the browser, but team collaboration and version control require Git integration.Databricks Repos lets you clone a Git repository directly inside the workspace and switch branches, Pull, Push, and view diffs from the UI.

This article covers what Repos is, supported Git providers, branching design, file format choices (.py vs .ipynb), secrets-management caveats, and integration patterns with CI/CD.

What is Databricks Repos?

Repos is a clone of a Git repository that appears as a folder inside the Databricks workspace. You do not need to install Git locally — branching, Pull, and Push all happen through the workspace UI.

The main characteristics of Repos are:

• Create and switch branches, Pull, Push, and view diffs from the workspace UI
• Notebooks, Python files, SQL files, and more can all be put under Git control
• Pull Requests are created and reviewed on the Git provider side (not in the Databricks UI)
• Each user can create multiple Repos folders (you can split per-branch if desired)

Supported Git Providers

Configure Git credentials under workspace User Settings → Git Integration. Credentials are stored per user and cannot be viewed by other users in the workspace.

Branching Best Practices

For team development with Databricks Repos, we recommend the following branching strategy.

A typical development flow looks like this:

1. A developer creates a feature/add-etl-pipeline branch in Repos
2. Develop and test code in the notebook UI
3. Commit and Push from the Repos UI
4. Open a Pull Request on the Git provider's web UI
5. Reviewers do the code review while CI runs the tests
6. After merge, the CI/CD pipeline (DABs, etc.) deploys to production

File Format Choice: .py vs .ipynb

The file format that Databricks Repos saves to Git directly affects the quality of Git management.

When you save a Databricks notebook in .py format, cell boundaries are represented by# COMMAND ---------- comments. The Databricks UI parses these comments to recreate the cell layout, so .py format preserves the full notebook UI experience.

# Databricks notebook source

# COMMAND ----------

from pyspark.sql import SparkSession

# COMMAND ----------

df = spark.read.format("delta").load("/mnt/data/sales")
display(df.limit(10))

# COMMAND ----------

# MAGIC %sql
# MAGIC SELECT region, SUM(amount) FROM sales GROUP BY region

Secrets Management Caveats

Never put secrets in Repos (i.e. in the Git repository). The rules below are non-negotiable.

• Never hard-code API keys, passwords, or tokens: they live forever in Git history and leak if the repo becomes public or a team member leaves
• Use Databricks Secret Scopes: dbutils.secrets.get(scope, key) retrieves secrets at runtime. Secret Scopes are stored securely in Databricks and never end up in Git.
• Use environment variables: inject secrets via cluster environment variables or Spark Config and reference only the key name from code
• Use .gitignore: keep config files and local .env files out of Git tracking

# BAD: Hard-coded secret
storage_key = "AbCdEfGhIjKlMnOp..."

# OK: Fetch from a Secret Scope
storage_key = dbutils.secrets.get(scope="my-scope", key="storage-key")

# OK: Fetch from Spark Config (injected via cluster settings)
storage_key = spark.conf.get("spark.custom.storage_key")

CI/CD Integration Patterns

The recommended division of labor is "Repos for development, CI/CD for deployment": Repos handles Git integration during development, while a CI/CD pipeline (DABs, GitHub Actions, etc.) handles deployment.

Repos Limitations

• The repo size limit is 10 GB (consider Git LFS for large datasets or binaries)
• One Repos folder maps to one Git repository (consider sparse checkout for monorepos)
• Merge conflict resolution beyond the basics must be done in a Git client
• Git Submodules are not supported
• Files in Repos folders are access-controlled via workspace ACLs (separate from Git permissions)

Repos vs Workspace Files

Databricks offers two places to store files: Repos and Workspace Files.

What the Exam Tests

• "What feature handles notebook version control?" → Databricks Repos (Git integration)
• "Can secrets be committed to the Git repository?" → No. Use a Secret Scope
• "How do you choose between Repos and DABs?" → Repos is for Git integration during development, DABs is for deployment automation
• "Where do you review PRs?" → On the Git provider's web UI (GitHub/GitLab, etc.)
• "What's wrong with managing .ipynb in Git?" → It's JSON, so diffs are hard to read and output cells become noise

Check Yourself

Frequently Asked Questions