A little over a year ago DataSense started looking into Snowflake as a new focus for implementing a next generation cloud data platform. As a company, we at DataSense focus on using Data Vault 2.0 for data integration, data engineering, and data analytics solutions. Crucial for a good Data Vault 2.0 implementation is a powerful database capable of handling the loads and transformations while being able to provide the necessary storage. Kent Graziano, Chief Technical Evangelist at Snowflake himself states that Snowflake provides an optimal environment for a Data Vault 2.0 project to thrive.
Now as we initially investigated Snowflake they promised much and delivered even more in their feature base. Doing away with a lot of the tedious maintenance that was required in data warehouses as we knew it back then. No need for fine tuning indexes, maintaining table statistics, cleaning up after queries or data loads. Their time travel feature was what any IT professional working with databases wished rollback was and query result caching was a massive performance increase when handling large tables.
The truth is Snowflake being a new software built from the ground up for cloud data warehousing meant it was fresh, modern and wasn’t plagued by many of the legacy issues or design philosophies holding established database providers back.
Here we are one year later and in this article we’d like to have a look at the progression of Snowflake in the year that’s gone by. As an IT company DataSense knows the importance of staying up to date with the latest developments and that stagnation in the world of IT is the same as regression. How well did Snowflake perform in the past year and what new developments have they come up with in that time frame? Please stay with us as we give you just a few of the improvements Snowflake made in the past year. We will be posting more elaborate use cases on the following features in de coming weeks.
Automated continuous data loading
Just this title alone might not seem like an innovative feature, however the implementation Snowflake chose is what makes it innovative. Using Snowpipe, the process of automated continuous loading of data is very easy to setup and maintain, whilst staying cost effective. You don’t need to bother yourself with figuring out the most efficient way to load your latest transactional data, you create a pipe and link it with a stage and table and Snowflake takes care of the rest.
It cannot be understated how important this is of a feature, as making use of the most recent and relevant data is incredibly important in the field of data analytics. When we’re talking about sales and financial decisions, the sooner we can acquire information and make reports based on it, the quicker we can make informed decisions and stay on top of any challenge that might present itself.
External functions
This new feature allows you to call code implementations developed and hosted outside of Snowflake and use them as if they were regular functions in Snowflake. This is an immensely powerful feature allowing you to reuse business logic unique to your company, trigger data loads and algorithms or call machine learning models that your data scientists have developed.
This new feature has a lot of so far untapped potential and although it’s currently limited to working with AWS API Gateway implementations the possibilities that are already there are game changing. It enables users to easily make use of complex logic that developers have implemented behind the scenes by making a simple function call in Snowflake.
Becoming the data cloud platform by sharing
From the very beginning Snowflake’s biggest asset was their secure data sharing functionality. By making use of this feature one can easily share any data they wish with other Snowflake data consumers. Even if the other party does not have a Snowflake account one can easily setup a Snowflake reader user for them.
Their biggest advancement on the development as a data sharing platform is the implementation of the data marketplace and the data exchange. The data marketplace provides users with a great platform to monetize their data, find more data to enhance their analytics with and potentially even generate new leads. Data exchange on the other hand offers the ability to set up a community of data consumers with whom you can share data and enhance or collaborate on projects together.
Education and certification
While it’s important to create new features, stay up-to date with the latest technological developments and to look towards the future, you can’t afford to neglect education. Towards the end of 2019 Snowflake introduced the SnowPro Core Certification and more recently they’ve added the SnowPro Advanced: Architect Certification. It’s important for any technology to have the ability to guarantee technological skills in the field and certification is a great step towards this.
Furthermore, they’ve created Snowflake University to help those on the road of acquiring certification and to train their customers and IT professionals in the optimal usage of Snowflake and implementing data warehouse projects on Snowflake. Snowflake university still needs some work in our minds, but it’s a great step in the right direction for the Snowflake team.
These are just four hand selected items that Snowflake has developed or massively improved on in the past year. This isn’t everything though, there are many different features not covered here and even simple adjustments to already existing features or the expansion of Snowflake to be available in new regions and on different hosting platforms such as Google Cloud. Snowflake has proven itself to be a company that has its eye on the future and isn’t looking to maintain the status quo, but to become the new status quo.
We’ll be covering the above features and many more in future articles showcasing the power of these features and Snowflake using actual use cases.
Overwhelmed by all the options and features Snowflake provides? Or looking to learn more about it and wondering how it could fit into your company’s technology stack? Please do contact us at DataSense to help you use Snowflake to turn your data into results.
You can reach us at www.datasense.be or via email info@datasense.be
