Day: January 9, 2018

ICYDK: I spend a lot of time futzing with editors and IDEs. To be honest, I’m pretty compulsive obsessive about the whole thing. I can watch YouTube videos for hours studying how each works and why people like them. One question that I really wanted to tackle is “Are there ways in which VS Code is actually better (i.e. more productive) than WebStorm, PyCharm, IntelliJ, etc." 

I think this video Moving from WebStorm to VSCode does a pretty good job tackling this question, but I remain unconvinced.  https://goo.gl/PxTjkY #DataIntegration #ML

ICYMI: ETL refers to extract, transform, load, and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in the last decade. With the arrival of new cloud-native tools and platforms, ETL is becoming obsolete. There are several emerging data trends that will define the future of ETL in 2018. A common theme across all these trends is to remove the complexity by simplifying data management as a whole. In 2018, we anticipate that ETL will either lose relevance or the ETL process will disintegrate and be consumed by new data architectures.

Unified Data Management Architecture

A unified data management (UDM) system combines the best of data warehouses, data lakes, and streaming without expensive and error-prone ETL. It offers reliability and performance of a data warehouse, real-time and low-latency characteristics of a streaming system, and scale and cost-efficiency of a data lake. More importantly, UDM utilizes a single storage backend with benefits of multiple storage systems which avoids moving data across systems hence avoiding data duplication and data consistency issues. Overall, it creates less complexity to deal with. https://goo.gl/mH3iKd #DataIntegration #ML

This is a series of stories I’m doing as part of my API Transit work, trying to map out a simple journey that some of my clients can take to rethink some of the basics of their API strategy. I’m using a subway map visual, and experience to help map out the journey, which I’m calling API transit–leveraging the verb form of transit, to describe what every API should go through.

Defining an API is the first stop of any API journey. When I say definitions, I’m not just talking about OpenAPI (fka Swagger), and, specifically, definitions for the surface area of your API. I’m talking about defining your idea, your goals, and the standard aspects of doing business with APIs. By API definitions, I mean having a robust toolbox of definitions for everything that is going into your API operations, from standardized dates and currencies to common data schema, and, yes, to making sure there is an active OpenAPI definition for every single one of your APIs. https://goo.gl/GJw6Yt #DataIntegration #ML