Day: May 16, 2018

ICYDK: We were exchanging emails with the APIMetrics team this week, discussing the challenges that come up when monitoring APIs. We are working through the different ways in which we can create real-time streams of data from monitors that APImetrics has set up, and one of the challenges you encounter when monitoring API infrastructure at scale is the ability to sift through large volumes of signals to understand what truly matters. When you are monitoring a large number of APIs, depending on the approach, the overall health of the API, and the clients that are consuming resources, you are likely to experience regular waves of HTTP status codes to sort through looking for the responses that actually matter and mean something. We are thinking about how cutting HTTP status code noise will make API monitoring streams more intelligent.

As we were discussing this, the topic of machine learning (ML) came up, and how we can use it to help make sense of the potentially huge amounts of HTTP status codes we’ll have to process. Allowing us to not respond to individual signals, but groups or patterns of signals, and make API monitoring a little more intelligent, and reduce the chances we’ll miss some important monitoring updates. Using machine learning to do the heavy lifting of looking through large amounts of data, and rely on humans to do the responding when relevant patterns are uncovered. If you rely on humans to look at every signal returned they are inevitably going to burn out, miss patterns, and become blind to much of what is returned. By leaning on ML to look through the volume of data, we can lighten the load for the human, and help them be a little more efficient in what they deliver. https://goo.gl/8yGBEA #DataIntegration #ML

* Hybrid approach: Semantic AI is the combination of methods derived from symbolic AI and statistical AI. Virtuously playing the AI piano means that for a given use case various stakeholders, not only data scientists, but also process owners or subject matter experts, choose from available methods and tools, and collaboratively develop workflows that are most likely a good fit to tackle the underlying problem. For example, one can combine entity extraction based on machine learning with text mining methods based on semantic knowledge graphs and related reasoning capabilities to achieve the optimal results.
* Data Quality: Semantically enriched data serves as a basis for better data quality and provides more options for feature extraction. This results in higher precision of prediction & classification calculated by machine learning algorithms. 
* Data as a Service: Linked data based on W3C Semantic Web standards can serve as an enterprise-wide data platform and helps to provide training data for machine learning in a more cost-efficient way. Instead of generating data sets per application or use case, high-quality data can be extracted from a knowledge graph or semantic data lake. Through this standards-based approach, also internal data and external data can be linked with little effort and can be used as a rich data set for any machine learning task. 
* Structured data meets text: Most machine learning algorithms work well either with text or with structured data, but those two types of data are rarely combined to serve as a whole. Semantic data models bridge the gaps between most used data formats such as XML, relational data, CSV or also unstructured text when using NLP and text mining methods. This allows us to link data across heterogeneous data sources to provide data objects as training data sets which are composed of information from structured data and text at the same time. 
* No black-box: In sharp contrast to AI technologies that ‘work like magic’, where only a few experts really understand the underlying techniques, Semantic AI seeks to provide an infrastructure to overcome information asymmetries between the developers of AI systems and other stakeholders, including consumers and policymakers. Semantic AI ultimately leads to AI governance that works on three layers: technically, ethically, and on the social and legal layer. 
* Towards self optimizing machines: Semantic AI is the next-generation Artificial Intelligence. Machine learning can help to extend knowledge graphs (e.g., through ‘corpus-based ontology learning’ or through graph mapping based on ‘spreading activation’) and, in return, knowledge graphs can help to improve ML algorithms (e.g., through ‘distant supervision’). This integrated approach ultimately leads to systems, which work like self optimizing machines after an initial setup phase, while being transparent to the underlying knowledge models. Graph Convolutional Networks (in progress) promise new insights.  https://goo.gl/bbVVCJ #DataScience #Cloud