Landing, Bronze, Silver, and Gold Layers Explained: A Practical Guide

If you have spent any time around Databricks or lakehouse architecture, you have probably heard the terms landing, bronze, silver, and gold thrown around. They sound like something out of a fantasy novel, but they are just a naming convention for the medallion architecture — a way of organising data as it flows from raw ingestion to business-ready tables.

In this article I will walk through what each layer actually means, how we build them using Spark SQL on Delta tables, and the practical things I have noticed while working with this pattern in real projects. We will keep it concrete — no buzzword bingo, just code and the reasoning behind it.

Why the Medallion Architecture Exists

Before we jump into the layers, it is worth understanding the problem this pattern solves. When you pull data from source systems — APIs, CDC streams, flat files dumped into S3 or GCS — it arrives in whatever shape the source decided. Column names are inconsistent, timestamps are in who-knows-what timezone, and half the rows are duplicates from a replay someone ran last Tuesday.

You could write a single massive pipeline that cleans everything in one go. I have seen teams do it. It works until it does not — the moment you need to debug a production issue at 3 AM, you will wish you had the raw data sitting somewhere untouched.

The medallion architecture breaks the pipeline into stages so each one has a single clear job. It is not about over-engineering. It is about being able to trace a bad number in the gold layer all the way back to the raw JSON file that caused it.

The Four Layers, In Order

Layer	Purpose	Typical Operations	Schema
Landing	Ingest as-is, keep exactly what the source sent	Copy or external table read	Schema-on-read or minimal
Bronze	Full history of source data, immutable append-only	Deduplicate, add metadata columns	Match source + audit columns
Silver	Cleaned, deduplicated, business-validated	Type casting, joins, filtering, enrichment	Curated, documented schema
Gold	Aggregated, business-facing tables	Aggregations, KPI calculations	Wide, denormalised as needed

Let us now go through each one with code.

Landing Layer — The Inbox

The landing layer is exactly what it sounds like — the place everything arrives. You do not transform anything here. If the source sends a CSV with 50 columns and 12 of them are empty, you keep all 50 columns. The whole point of landing is that you can always replay from the raw source if something downstream goes wrong.

In practice, landing is often an S3 bucket, a GCS bucket, or an ADLS container. You might not even use Delta tables here — sometimes it is just raw Parquet or JSON files sitting in object storage.

-- Example: reading raw JSON from a landing location
CREATE OR REPLACE TEMP VIEW landing_orders_raw AS
SELECT * FROM json.`s3://my-bucket/landing/orders/2025/10/07/*.json`;

The landing layer is not where you do anything clever. You just make sure the files are there and are readable. If a file is corrupted or the schema changed from the source, you want to catch it in the next layer, not here.

Bronze Layer — The Source of Truth

Bronze is the first Delta table you write. It stores the data exactly as it arrived, plus a few metadata columns that help you trace lineage. The key rule: bronze tables are append-only. You never UPDATE or DELETE rows in bronze. Every new batch of data gets appended. If the same record appears in two different ingestion runs, bronze keeps both.

Why append-only? Because if your deduplication logic in silver has a bug, you can always go back to bronze and re-read the full raw history. I have been saved by this more than once.

-- Creating a bronze table from landing data
CREATE OR REPLACE TABLE bronze.orders USING DELTA
LOCATION 's3://my-bucket/bronze/orders'
AS
SELECT
  *,
  current_timestamp() AS ingested_at,
  input_file_name() AS source_file,
  'orders-api' AS source_system
FROM landing_orders_raw;

A few things to watch out for in bronze:

• Schema evolution: If the source adds a column tomorrow, your bronze table needs to handle it. Use mergeSchema or set spark.databricks.delta.schema.autoMerge.enabled = true if you know the change is safe.
• Partitioning: Do not go overboard. Partitioning by ingestion date is usually enough. Over-partitioning bronze tables will hurt write performance and you will rarely query bronze except during debugging.
• Small files: If you are ingesting streaming data every minute, you will end up with thousands of tiny files. Run OPTIMIZE on the bronze table periodically, or better, use Auto Optimize on Databricks.

Silver Layer — Where the Real Work Happens

Silver is where you clean, standardise, deduplicate, and join. This is the layer where data becomes usable for analysts and data scientists. The schema here should be well-documented and stable — if someone asks “what does the orders table look like?” they are asking about silver.

-- Silver transformation: deduplicate, cast types, drop junk columns
CREATE OR REPLACE TABLE silver.orders USING DELTA
LOCATION 's3://my-bucket/silver/orders'
AS
WITH deduped AS (
  SELECT *,
    ROW_NUMBER() OVER (
      PARTITION BY order_id
      ORDER BY ingested_at DESC
    ) AS rn
  FROM bronze.orders
  WHERE order_date IS NOT NULL
)
SELECT
  CAST(order_id AS STRING) AS order_id,
  CAST(customer_id AS STRING) AS customer_id,
  CAST(order_amount AS DECIMAL(12,2)) AS order_amount,
  TO_TIMESTAMP(order_date, 'yyyy-MM-dd HH:mm:ss') AS order_date_utc,
  CASE
    WHEN order_status IN ('COMPLETED', 'SHIPPED', 'DELIVERED') THEN order_status
    ELSE 'OTHER'
  END AS order_status_clean,
  ingested_at,
  source_file
FROM deduped
WHERE rn = 1;

Things I have learned about silver the hard way:

• Deduplication is trickier than it looks: Deciding which row to keep (latest, earliest, the one with more non-null columns) depends on the business context. Do not assume latest is always correct.
• Null handling creeps up: Source systems love sending empty strings where you expect NULL. A simple NULLIF(col, '') in silver saves confusion in gold.
• Join keys need validation: If you join orders to customers on customer_id, make sure the key actually matches. I have seen production tables where 30% of rows dropped silently because of a whitespace difference in the join key.
• Keep it idempotent: Silver pipelines should be safe to re-run. If you run the same silver job twice on the same bronze data, it should produce the same result.

Production Considerations for Silver

In a real production setup, you would not do a CREATE OR REPLACE TABLE every time. That approach works for a demo or a small dataset, but if your silver table has 500 million rows, you do not want to rewrite the whole thing nightly. Instead you would:

• Use incremental processing: read only new records from bronze (using a watermark on ingested_at) and MERGE them into silver.
• Use Change Data Feed (CDF) on bronze to detect which rows changed without scanning the whole table.
• Set up table constraints on silver columns (NOT NULL, CHECK) so bad data fails at write time, not six hours later when a dashboard breaks.

Gold Layer — Business-Ready Tables

Gold is the layer your stakeholders care about. These are aggregated tables, KPI summaries, and denormalised views built for specific use cases — a dashboard, a machine learning model, or a weekly email report. Gold tables should be dead simple to query. Someone with basic SQL should be able to write SELECT * FROM gold.daily_sales_by_region and get exactly what they expect.

-- Gold: daily sales aggregation
CREATE OR REPLACE TABLE gold.daily_sales_summary USING DELTA
LOCATION 's3://my-bucket/gold/daily_sales_summary'
AS
SELECT
  DATE(order_date_utc) AS order_day,
  order_status_clean,
  COUNT(DISTINCT order_id) AS total_orders,
  COUNT(DISTINCT customer_id) AS unique_customers,
  SUM(order_amount) AS total_revenue,
  AVG(order_amount) AS avg_order_value
FROM silver.orders
GROUP BY 1, 2;

Gold is also where you want to think about performance. Add Z-ordering on columns that get filtered often. Use Liquid Clustering if you are on a recent Databricks runtime. Partition by date for time-series queries.

Some practical notes on gold:

• One gold table per use case: Do not try to build a single do-everything gold table. You end up with 80 columns and nobody knows what half of them mean. Build specific tables for specific needs.
• Naming matters: Call it daily_sales_summary, not ds_gld_v2_final_final. Future you (and your colleagues) will thank you.
• Expectation management: Gold tables are derived from silver, which is derived from bronze. If the source data has missing orders for Tuesday, your gold dashboard will show wrong numbers. Stakeholders need to understand this dependency chain.

When to Skip a Layer

Not every pipeline needs all four layers. Here are some scenarios:

• Kafka streaming with a schema registry: Your source data is already structured and validated. You might skip bronze and write straight to silver.
• Small, stable CSV files that never change: A one-off import of a 500-row reference table does not need a full bronze-silver-gold flow. Load it directly.
• Landing and bronze can merge: If you are streaming into a Delta table with Auto Loader, the landing layer is implicit — the source files in object storage act as your landing.

The point of the architecture is not to add layers for the sake of it. It is to give you a way to trace data lineage and recover from problems. If your pipeline is simple and you have another mechanism for replay and debugging, fewer layers are fine.

Wrapping Up

The medallion architecture is a sensible way to organise data pipelines. It is not a silver bullet, and it does not replace thinking about your actual data model. But if you follow the pattern — raw in landing, full history in bronze, cleaned and deduplicated in silver, business-ready in gold — you will have a much easier time debugging issues, onboarding new team members, and keeping your data trustworthy.

Start with bronze and silver. Get the deduplication right. Gold will follow naturally.