Getting Started with Unity Catalog: A Practical Guide for Data Teams

When our team first migrated from the default Hive metastore in Databricks to Unity Catalog, I’ll be honest — it wasn’t something any of us were excited about. Governance tools rarely are. But a few months in, the benefits became clear: we could finally see who had access to what, grant permissions without copy-pasting SQL across three workspaces, and stop waking up to “who deleted that table?” Slack messages.

This article walks through what Unity Catalog actually is, how to set it up from scratch, and what I wish someone had told me before we started. If you’re a data engineer who just got handed a ticket saying “enable Unity Catalog for the team”, this is for you.

What Is Unity Catalog and Why Should You Care?

Unity Catalog is Databricks’ answer to centralized data governance. Before Unity Catalog, each Databricks workspace had its own Hive metastore. That meant if you had three workspaces — dev, staging, prod — you had three separate sets of table metadata, three sets of permissions, and no easy way to know if the sales.transactions table in dev was the same as the one in prod.

Unity Catalog fixes this by providing a single metastore that spans workspaces within a region. Tables, views, volumes, and models are all registered in one place. Permissions are managed centrally using SQL GRANT statements. And lineage tracking is built-in — not something you bolt on later.

Here’s the rough hierarchy:

• Metastore — the top-level container, one per region
• Catalog — like a database in traditional systems, groups schemas
• Schema — like a database schema, groups tables and views
• Table / View / Volume / Model — the actual data objects

The big shift mentally is that security is now at the catalog and schema level, not just the table level. Once you wrap your head around that, the rest falls into place.

Before You Start: What You Need

Not every Databricks workspace can use Unity Catalog. Here’s what’s required:

Requirement	Details
Databricks account	Must be on the Premium plan or above
Workspace	Attached to the account, not a standalone workspace
Storage location	An AWS S3 bucket or Azure ADLS Gen2 container for managed storage
Account admin access	You need to be an account admin to create the metastore
Unity Catalog-enabled cluster	Use Shared or Single User clusters with Unity Catalog enabled

If your workspace was created before Unity Catalog was GA (mid-2022), you might need to enable it explicitly. Check your admin console under “Metastores”.

Step 1: Create the Metastore

This is a one-time setup done at the account level.

Go to the Databricks Account Console (not the workspace), navigate to Catalog under the data section, and click Create metastore. You’ll need:

• A name (something like prod-metastore-us-east)
• The region (must match where your workspaces live)
• A storage location — the S3 bucket or ADLS container where managed tables will live

-- Example metastore configuration (conceptual)
-- This is done through the UI, not SQL
-- Name: company-metastore
-- Region: us-east-1
-- Storage: s3://databricks-unity-catalog-metastore/

Once the metastore is created, attach your workspaces to it. Each workspace can only belong to one metastore. If you have multiple workspaces in the same region, they should all attach to the same metastore — that’s the whole point.

One thing I tripped over: if you have workspaces with existing Hive metastore tables, those don’t automatically migrate. You’ll see both the hive_metastore catalog and your new Unity Catalog catalogs side by side until you explicitly upgrade those tables.

Step 2: Create Your First Catalog and Schema

Now switch to a workspace. Open the SQL editor and create a catalog:

CREATE CATALOG IF NOT EXISTS sales_data
COMMENT 'Production sales data catalog';

Then a schema inside it:

CREATE SCHEMA IF NOT EXISTS sales_data.transactions
COMMENT 'Payment transactions from all channels';

At this point you have sales_data.transactions as a namespace. You can start creating tables under it:

CREATE TABLE sales_data.transactions.online_orders (
  order_id STRING,
  customer_id STRING,
  amount DECIMAL(10, 2),
  order_date DATE
)
USING DELTA
LOCATION 's3://my-data-lake/sales/online_orders/';

A few things to notice:

• The three-level naming (catalog.schema.table) is mandatory in Unity Catalog — no more bare transactions table names
• The LOCATION is optional; if you omit it, Unity Catalog uses the metastore’s managed storage
• Tables are Delta by default, which is exactly what you want

Step 3: Managing Permissions (Where Most People Get Stuck)

Permissions in Unity Catalog follow a simple model: you grant privileges on securable objects to principals. But the hierarchy matters more than you’d expect.

Here are the key permissions:

Privilege	What it allows
USE CATALOG	Needed to read/write anything inside the catalog
USE SCHEMA	Needed to access tables in that schema
SELECT	Read from a table or view
MODIFY	Insert, update, delete, merge on a table
CREATE TABLE	Create tables in a schema
READ VOLUME	Access files in a volume

Here’s a typical setup for a data analyst who needs read-only access:

-- Grant catalog access
GRANT USE CATALOG ON CATALOG sales_data TO `analyst-team@company.com`;

-- Grant schema access
GRANT USE SCHEMA ON SCHEMA sales_data.transactions TO `analyst-team@company.com`;

-- Grant table access
GRANT SELECT ON TABLE sales_data.transactions.online_orders TO `analyst-team@company.com`;

And for a data engineer who needs to create and write:

GRANT USE CATALOG ON CATALOG sales_data TO `eng-team@company.com`;
GRANT USE SCHEMA ON SCHEMA sales_data.transactions TO `eng-team@company.com`;
GRANT CREATE TABLE ON SCHEMA sales_data.transactions TO `eng-team@company.com`;
GRANT MODIFY ON TABLE sales_data.transactions.online_orders TO `eng-team@company.com`;

The thing that threw me off initially is that USE CATALOG and USE SCHEMA are separate. Even if you have SELECT on a table, you’ll get an error if you haven’t been granted USE SCHEMA on the schema and USE CATALOG on the catalog. The permissions don’t cascade automatically.

Groups are your friend here. Instead of granting permissions to individual users, create account groups and assign permissions to those. When someone joins or leaves the team, you update the group membership, not fifty GRANT statements.

Step 4: Working with External Locations

If your data lives in S3 or ADLS outside the managed storage, you need to register external locations. This is another account-level setup:

1. Go to the Account Console → Catalog → External Locations
2. Click Create and point it to your S3 bucket or ADLS container
3. You’ll need to provide IAM or service principal credentials so Databricks can access the storage

Once registered, you reference these locations when creating external tables. Without a registered external location, Unity Catalog won’t let you create a table pointing to that S3 path — it simply rejects the command.

-- This will fail without a registered external location
CREATE TABLE sales_data.transactions.offline_orders
LOCATION 's3://unregistered-bucket/data/';

-- This works because the location is registered
CREATE TABLE sales_data.transactions.offline_orders
LOCATION 's3://registered-bucket/data/';

Things to Be Careful About

Existing Hive metastore tables. Unity Catalog doesn’t automatically upgrade them. You’ll see two catalogs — hive_metastore and whatever you created. To upgrade a table, you can use SYNC commands or rebuild it under a Unity Catalog path. Plan for this migration time.

Row filters and column masks. Unity Catalog supports row-level and column-level security through row filters and column masks. They’re powerful but easy to misconfigure. If you apply a row filter that restricts a table to rows where region = 'US', suddenly anyone without US data sees zero rows — and there’s no visible error, just empty results. Test these with a non-admin user before rolling out.

Data lineage catches dependencies you forgot about. When you enable lineage tracking, Unity Catalog catalogs every table a notebook reads and writes. This is great for impact analysis, but it can be surprising when you see just how many downstream jobs touch a single source table. In a production setting, plan for someone to own the review of lineage gaps — tables that are read but shouldn’t be, or pipelines that bypass cataloged sources entirely.

Not all cluster types work. High Concurrency clusters with table access control enabled don’t play nicely with Unity Catalog. Use Shared access mode clusters instead. This tripped us up when we tried to run existing notebooks on Unity Catalog-enabled workspaces and got baffling permission errors.

What Changes in Production

In a simple demo, you create one catalog, one schema, grant a few permissions, and call it done. In production, here’s what changes:

• Catalog design matters. Don’t create one giant catalog called production with 200 schemas. Split by domain — sales_data, marketing_data, finance_data — and manage each with separate permission groups. It’s easier to grant USE CATALOG on sales_data to the sales team than to grant USE SCHEMA on 30 individual schemas.
• Use service principals for jobs. When a Databricks job reads from or writes to Unity Catalog tables, don’t attach a human’s identity to the job. Create a service principal, grant it the minimal permissions needed, and configure the job to run as that principal.
• Terraform it. Once you’ve validated the setup manually, codify it. The Databricks Terraform provider has full support for Unity Catalog resources — databricks_metastore, databricks_catalog, databricks_schema, databricks_grants. Manual click-ops doesn’t scale past three schemas.
• Have a naming convention. Three-level names are verbose. Agree on catalog names that are short and descriptive. sales_data.transactions.online_orders is better than sales_data_department.transactional_schema.online_order_transactions_table.

Wrapping Up

Unity Catalog is one of those tools that feels like overhead when you first set it up and like a safety net once it’s in place. The time you spend configuring permissions and external locations upfront pays off the first time someone asks “who has access to this table?” and you can answer with a single SHOW GRANTS statement instead of digging through IAM policies, workspace settings, and three different config files.

Start small — one catalog, one schema, and a clear plan for what to migrate next. Don’t try to move everything at once. And if you’re using Terraform for the rest of your infrastructure, Unity Catalog resources belong in that same repository.