Author: admin

#ICYDK: I run a lot of different applications that depend on GitHub and that use GitHub authentication as the identity and access management layer for these apps. One of the things I like the most about GitHub and how I feel it handles it’s OAuth more thoroughly than most other platforms is how they let you get your own OAuth token under your settings > developer settings > personal access tokens. You don’t need to set up an application and do the whole OAuth dance, you just get a token that you can use to pass along with each API call.

I operate my own OAuth server, which allows me to authenticate using OAuth with many leading APIs, so generating an OAuth token and setting up a new provider isn’t too hard. However, it is always much easier to go under my account settings, create a new personal access token for a specific purpose, and get to work playing with an API. I wish that ALL API providers did this. At first glance, it looks like GitLab, Harvest, TypeForm, and ContentFul all provide personal access tokens as a first option for on-boarding with their APIs. Demonstrating this is more of a pattern than just a GitHub feature. https://goo.gl/gnCd75 #DataIntegration #ML

We’ve been developing more serverless applications that run on AWS lately, which let you stream data from a variety of API providers into your Amazon S3 data lake(s). Continuing with the same theme of building data lakes from 3rd party APIs, we wanted to leverage the GitHub platform for building data lakes by streaming data from 3rd party APIs, as a simple JavaScript package that runs entirely on GitHub — pushing the boundaries of what is possible with APIs, GitHub, and data lakes. As part of this work, we’ve published our first streaming topic subscriptions using Reddit to GitHub. This JavaScript micro application runs on GitHub, hosted by GitHub pages, and leverage Jekyll, HTML, CSS, and JavaScript to deliver the application interface. The micro application connects to the Reddit API, allows for five separate searches that are defined using OpenAPI, which are then proxied using Streamdat.io delivering Server-Sent Events (SSE) streams for each topic, which gets displayed on the screen at the bottom of the page. Providing five separate topical streams from Reddit that can be triggered and monitored via the browser on the GitHub Pages hosted web application.

Reddit allows for API searches without being authenticated, but if you authenticate using OAuth, they’ll let you make more API calls, which allows for longer running streams. Additionally, if you authenticate with GitHub, it will allow you to save the topical stream to a GitHub repository stored within your GitHub account. The feature requires repository level access to your GitHub to turn on this feature and allow for data to be saved to the repository, but it is something that opens up an entirely new way for developing data lakes, that can then be put to work using Git, or with the GitHub API. Streaming in data from a variety of 3rd party API sources, and storing it within private GitHub repositories, for use in a variety of applications, and the training of machine learning models. https://goo.gl/q4duWL