Matillion ETL is a cloud ELT solution purpose-built with cloud data warehouses like Snowflake, Azure Synapse and Google BigQuery in mind, with a separate custom version for each provider. The cloud native design with all data jobs running in your cloud environment offers maximum speed, scalability, and efficiency and is optimized for both resource and [...]
At DataSense, we believe building a data platform always starts with a well-defined plan. Custom-built solutions tend to be difficult to maintain and are especially not future proof. Therefore, we like to work with an open standard, namely Data Vault 2.0. This standard combines methodology with a modelling approach and architectural guidelines. Automation In Data Vault 2.0, [...]
General Metadata management is the management of data about other data. It comprises, among other things, data modeling, data governance and metadata administration. In this piece, we will focus on data lineage, a key component in metadata management. Data lineage describes the complete data life cycle by providing insight in the data flow from start [...]
Challenge Companies dispose of enormous amounts of data, ranging from operational data over financial data, to benchmarks, social media data, … This data is then transformed into comprehensive reports to inform strategic decision-making processes. To ensure the accuracy and reliability that is expected of these reports, we must first ensure that the data itself is [...]
Challenge Companies collect enormous amounts of data over time going from operational data such as stock values, supplier data, orders,.. to financial data such as pricing, payments, bank statements, accounting,.. and even external information such as demographics, benchmarking data, social media,….Only a few legacy software companies supply an all-in-one package. These packages may give you [...]
Challenge Usually, business stakeholders are no data experts. They are primarily interested in getting valuable insights out of data, upon which they can take the appropriate actions. As a result, it is difficult for stakeholders to know what is possible until an analyst shows them. Furthermore, they have no idea about the effort it requires [...]
Through the years, we have learned from implementations at various customers that setting up and maintaining a Snowflake environment can be time consuming and cumbersome. Especially in a fast growing environment with a lot of users, making sure every user has the correct privileges can quickly become very complex. Typically, this results in a lot of extensive [...]
Snowflake has numerous advantages and strategic values who all lead to the success we know and witness today. In the last year, Snowflake has become an established value as a cloud data platform thanks to an amazing marketing campaign but moreover, by one of the most successful IPOs in the history of IT. Before going [...]
In the modern IT landscape data is more important than ever, yet when sharing data, we are still stuck using the same outdated methods. Because why are we still driving from place to place to swap out hard drives with a database dump on it? Some organizations and/or sectors wish to provide their business partners [...]
A little over a year ago DataSense started looking into Snowflake as a new focus for implementing a next generation cloud data platform. As a company, we at DataSense focus on using Data Vault 2.0 for data integration, data engineering, and data analytics solutions. Crucial for a good Data Vault 2.0 implementation is a powerful database capable of handling the loads [...]
