Multiple Speed Approach
- Business requirements:
Business requirements will change over time because today’s use case differs from the one within a day, month, year,… That’s why the consumption layer should be resilient to change. By dividing the data hub into two layers, the consumption layer can change whenever the business wants because all data will be kept in the integration layer without changing it by applying business rules.
- Divide and conquer:
- When implementing the data hub, we split up the team in two sub teams. One team will focus on integrating data entities coming from different data sources. All functional relevant data will be integrated, even if the end user does not need this data at that moment. This process is automated as much as possible by applying data warehouse automation, continuous integration and continuous delivery (CI/CD). This approach offers organizations scalability and flexibility in adding new data sources and new data entities of existing data sources.
- The other team will focus on the transformation of the raw data (stored in the integration layer) into valuable data products which will be used by business stakeholders. When requirements change and f.e. new data is needed in the consumption layer, it can be sourced from the integration layer because all functional relevant data is present in this layer.