Unity Catalog

The Unity Catalog is a unified governance solution for all data assets in Databricks. It provides a centralized view of all data assets and allows us to manage access to data at scale. Unity Catalog is designed to simplify data management and governance across multiple cloud environments, making it easier to secure and share data while ensuring compliance with regulatory requirements.

We use this to organize our data, models and files as well as managing permissions and metadata associated with our data assets.

Key Features of Unity Catalog

• Centralized Data Governance: Unity Catalog provides a single interface for managing data assets across multiple cloud environments, making it easier to enforce data governance policies and ensure compliance with regulatory requirements.
• Fine-Grained Access Control: Unity Catalog allows for fine-grained access control at the table, column, and row levels, enabling organizations to enforce data security policies and ensure that sensitive data is only accessible to authorized users.
• Data Lineage: Unity Catalog provides data lineage capabilities, allowing organizations to track the origin and transformation of data assets, which is essential for compliance and auditing purposes.
• Metadata Management: Unity Catalog provides a centralized repository for managing metadata associated with data assets, making it easier to discover, understand, and use data across the organization.
• Integration with Databricks: Unity Catalog is fully integrated with Databricks, allowing organizations to leverage the power of Databricks for data processing and analytics while maintaining a unified view of their data assets.

Unity Catalog Structure

Our Unity Catalog is undergoing some changes to better align with our data organization principles. See this document for more information on the new structure.