In one of our previous blogposts we discussed one of the most vital tools in the modern data stack, dbt. Once you have chosen to use dbt there are two questions that remains, which version should I use and what is the difference between the two? In this blogpost we will dive into the differences and similarities between the two versions and how you can leverage them within your data stack.

// DBT CORE

dbt Core is an open-source command line tool available through the Python pip package installer that assists data engineers and analysts in handling data warehouse transformations. It is installed and utilized through the command line, which may be familiar to some but daunting for others. Each supported data warehouse environment has its own specific version of the dbt Core package, Snowflake for example uses the dbt-snowflake package. Each specific package contains all the code needed for dbt to interact with the chosen data warehouse.

One of the main advantages of using dbt Core is the expanded selection of supported platforms. Where dbt Cloud only supports the official dbt adapters, dbt Core gives you access to the community adapters which include popular platforms like Azure Synapse, SQLServer and Oracle. So, if you want to leverage the power of dbt on one of those platforms, Core is the only way to go.

All of the main dbt functionalities from integrated testing to model building and documentation are provided through different dbt commands like “dbt test” and “dbt run –select my_model+”. Featurewise there is no disadvantage in using Core instead of Cloud. The main difference between the two is in the development experience itself. dbt Core doesn’t provide an IDE like the one in dbt Cloud so you will need to provide your own. Luckily developing in dbt is as simple as writing SQL statements and YAML, so you can use any text editor you like. However, our preference goes to Visual Studio Code because it offers the dbt Power User extension. This extensions mimics some of the functionalities of the dbt Cloud web IDE, like visualizing model results, and offers code completion and snippets to enhance the development experience.

dbt Core

dbt Cloud

Open source, free
Community adapters (SQLserver, Synapse,..)
All dbt features
No user interface, only CLI

User friendly interface for easier development
Only official adapters
Job scheduling
Hosted documentation, lineage preview

// DBT CLOUD

While dbt Core features all the dbt functionalities you need, it requires a bit more technical knowledge and skill to use to its fullest. This is where dbt cloud comes in.

dbt Cloud is a pre-configured, fully managed service that operates on dbt Core’s engine. Although it’s not necessary to have dbt Cloud to use dbt, it streamlines the process and gets you operational quickly and effortlessly.

The biggest selling point of dbt Cloud is the cloud IDE that makes it easy to develop and collaborate with all your team members. The dbt Cloud IDE is a one-stop-shop where you can write and run your dbt models. You can visualize the relations between your models in real-time while developing and preview what the results of a run would look like.

On top of that the IDE features auto-complete, documentation hosting and an integrated git environment simplifying your entire development process. Besides streamlining your development process, the cloud environment also offers features like job scheduling and easy CI/CD integration to enable teams to deliver data products at faster speeds.

// CONCLUSION

While DBT Core features all the DBT functionalities you need, it requires a bit more technical knowledge and skill to use to its fullest. This is where DBT cloud comes in:

DBT Cloud is a pre-configured, fully managed service that operates on DBT Core’s engine
Streamlines the process and gets you operational quickly and effortlessly
The cloud IDE that makes it easy to develop and collaborate with all your team members

The DBT Cloud IDE is a one-stop-shop where you can write and run your DBT models:

Visualize the relations between your models in real-time while developing and preview what the results of a run would look like
Auto-complete
Documentation hosting
Integrated git environment simplifying your entire development process

Besides streamlining your development process, the cloud environment also offers features like: