ETL refers to extract, transform, load, and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in the last decade. With the arrival of new cloud-native tools and platforms, ETL is becoming obsolete. There are several emerging data trends that will define the future of ETL in 2018. A common theme across all these trends is to remove the complexity by simplifying data management as a whole. In 2018, we anticipate that ETL will either lose relevance or the ETL process will disintegrate and be consumed by new data architectures.
Unified Data Management Architecture
A unified data management (UDM) system combines the best of data warehouses, data lakes, and streaming without expensive and error-prone ETL. It offers reliability and performance of a data warehouse, real-time and low-latency characteristics of a streaming system, and scale and cost-efficiency of a data lake. More importantly, UDM utilizes a single storage backend with benefits of multiple storage systems which avoids moving data across systems hence avoiding data duplication and data consistency issues. Overall, it creates less complexity to deal with. https://goo.gl/uDD7u1 #DataIntegration #ML
