We’ve already established the synergy of dbt & Databricks. However, adding data warehouse automation to the mix might be the missing link in the story. The combination of dbt & databricks provides a robust solution for streamlining data workflows and enhancing data lakehouse architectures. It leverages Databricks’ versatility in data processing and machine learning with [...]

As data, and big data in particular, becomes more prevalent, so too do tools for storing, analysing, and transforming it. And thanks to the constantly evolving data landscape of the last few years, there has been a rush of new big data platforms that offer all-in-one solutions. The most recent of these platforms is Microsoft [...]

Scaling beyond limits dbt, or data build tool, has become the cornerstone for organizations managing data transformations. From young start-ups to large corporations, dbt serves as the centerpiece for orchestrating data workflows. However, as organizations grow, the challenge of coordinating multiple requirements within a single dbt project becomes apparent. The solution? dbt Mesh, a pattern [...]

Databricks Unity Catalog is a pioneering unified governance tool designed for managing data and AI on the Lakehouse. Unity Catalog is Databricks’ answer to Data Governance challenges. It assists organizations in maintaining control over all data assets, managing data access, ensuring data quality, and the lineage. // WHAT Databricks Unity Catalog offers the following capabilities: [...]

Data Streaming refers to the continuous process of transmitting, processing, and analyzing data in real-time as it is generated or received. It involves the constant flow of data from its source to its destination, where it can be processed, analyzed, and acted upon without significant delay, typically within milliseconds to seconds. Real-time data streaming is [...]

dbtLabs made several impressing new releases at the recent Coalesce 2023 conference. One of them is the dbt Explorer.  This functionality allows you to have a clear and visually pleasing overview of your different dbt projects, all elements within them and the different access levels.