Day: March 10, 2018

ICYMI: ​Employers from over 40 countries will soon be able to post jobs and receive applications through Facebook, the company announced. The global expansion to countries such as Argentina, Brazil, Italy, Spain and the U.K. will cut into the traditional job board market, especially reducing posts from small and local businesses. https://goo.gl/ZnLwqV #GlobalHR #HRTech

The gig economy is growing at a record pace. BLS data shows that roughly 34% of the workforce is composed of independent contractors. The number of freelancers is expected to rise at least 60% between 2017 and 2020. Both freelancers and the brands depending on them are grappling with a variety of challenges. Blockchain is key to addressing some of the most pressing problems.

How can blockchain benefit the gig economy?

Blockchain was originally developed to process and authenticate digital currency transactions. It was the official ledger for bitcoin.

The same technology can be repurposed for managing online contracts. Here are some of the key advantages.

Verifying freelancer work

Brands must work with reputable freelancers to meet their business goals. Unfortunately, it can be difficult to verify the authenticity of their work and establish their reputation.

Blockchain provides a cutting-edge solution for both parties. Blockchain ledger systems enable freelancers to create unique online identities that nobody else has the cryptographic keys to access. This allows them to verify their identity and ownership of their content to protect against impersonators.

Protection and transfer of intellectual property rights

The Internet has complicated the enforcement of intellectual property rights. In the United States, United Kingdom and many other developed countries, originators of written content aren’t required to register their IP with a government registry. They secure their copyrights as soon as their content is published.

However, verifying the original creator of the content over the Internet is very difficult. Timestamps can’t be easily accessed, unless the content has been saved by a trusted Internet archiver.

Blockchain offers an ideal solution. Content developers can register their work with a blockchain registry such as Bernstein, so there will be no dispute over ownership of the IP rights.

Cost-effective alternatives to escrow

Traditional gig economy platforms charge fees ranging from 10 to 15% of every contract. These fees have raised the cost structure of the gig economy.

Blockchain provides a much more cost-effective alternative. Freelancers and clients can enter into contracts with clearly defined payment terms. The blockchain can serve as an intermediary between the two and ensure transactions are processed as soon as the work is completed to the client’s satisfaction. Transaction fees via the blockchain may be as low as 0.5%. This enables freelancers to lower their rates and split the savings between them and their clients.

New work agreement platforms like Grain.io are being built on blockchain to process work agreement securely and at a much lower cost. Former Senior Director at Microsoft and Grain CEO Onno Hektor explains, “By moving work agreements to a blockchain solution, we can substantially lower overhead costs and make both sides of the transaction better off. Transferring money to workers becomes cheap by eliminating unfavorable exchange rates or high transaction costs.”

Toolkit for blockchain developers offering gig economy solutions

Blockchain developers can take advantage of a number of development tools to create cutting-edge solutions for the gig economy. A few years ago, their only option was to use bitcoin to create daaps. Since bitcoin is intended for peer to peer transactions, it isn’t adequate solution for daap development.

Here are some alternative watching development tools.

Mist

Mist is a blockchain solution that blockchain developers use to handle a wide range of applications. It is commonly used for awarding and regulating smart contracts, which makes it an ideal solution for gig economy platforms.

Etherium blockchain developers that haven’t used Mist before can find it on Github. This environment has a wide array of scripts, modules and other resources they need to begin coding.

BaaS

BaaS is an Etherium blockchain developer platform created by Microsoft Azure. All content hosted and shared through BaaS is encrypted.

BaaS has a permanent digital transaction trail. If developers need to correct any transactions, a rectifying transaction must be completed. Transactions cannot be modified or deleted.

One of the key benefits of the BaaS environment is its machine learning capabilities. The environment is interconnected with other developers, so it becomes more streamlined as the transaction ledger scales. Replication of transactions also continually enhances the authenticity of the BaaS environment.

Blockchain Testnet

Blockchain testnets simulate the environment of the actual blockchain, while keeping it sanitized from test transactions. There are a number of different Blockchain testnets, but Blockcypher is one of the most popular.

With Blockcypher, blockchain developers begin the simulation the blockchain environment by creating a push transaction. This is accomplished by selecting the transaction type and entering the hexadecimal code associated with it. After the simulated transaction is created, can be publicly broadcast to the blockchain testing environment.

The process can be reversed to decode blockchain transactions as well. http://bit.ly/2txEHho #SAP #SAPCloud #AI

ICYDK: Apache Camel empowers you to define routing and mediation rules in a variety of domain-specific languages, including a Java-based Fluent API, Spring or BlueprintXML Configuration files, and a Scala DSL. It also uses URIs to work directly with any kind of Transport or messaging model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF, as well as pluggable Components and Data Format options. Apache Camel is a small library with minimal dependencies for easy embedding in any Java application.

A few weeks ago, I blogged about Apache Camel URI completion in the Eclipse XML Editor. This week I’m happy to announce that the same feature is available for 2 others IDEs: VS Code and Eclipse Che. https://goo.gl/XGWJ2m #DataIntegration #ML

In this blog, we’ll discuss Text Mining Functions.  Functions available to find top ranked related and relevant documents and terms.

Figure 1 shows the permutations and combinations available for doing Text Mining.

Figure 1: Text Mining Functionality

Available Functions in SAP HANA Text Mining

* First Block is to identify related and suggested terms. Functions available for these operations

* TM_GET_RELATED_TERMS
* TM_GET_SUGGESTED_TERMS

* Second Block is to identify relevant or similar documents. Function available for this operation

* TM_GET_RELATED_DOCUMENTS
* TM_GET_RELEVANT_DOCUMENTS

* Third Block is to identify relevant terms of a documents. Function available for this operation

* TM_GET_RELEVANT_TERMS

* Fourth Block is to categorize or classify documents. Function available for this operation

* TM_CATEGORIZE_KNN

*************************************************************************

Document Functions

TM_GET_RELATED_DOCUMENTS

This text mining function returns the top-ranked related documents for a query document within a search request and stores these documents (including metadata) in the return table. Syntax: TM_GET_RELATED_DOCUMENTS ( ) where := DOCUMENT { [ LANGUAGE ] [ MIME TYPE ] | ( ) [ LANGUAGE ] [ MIME TYPE ] | IN FULLTEXT INDEX WHERE } Either provide text as string or provide a select query or specify query document part of full text index using where clause for restriction. := SEARCH FROM [ WHERE ] [ WITH TERM TYPE , … ] Specifies the set of reference documents by specifying and . Specified reference column must be of type TEXT or must have full text index. To restrict the set of reference documents to be used in calculations specify where condition. Further restriction can be introduced using WITH TERM TYPE example ‘proper*’,’noun’ in which will only consider the proper names or nouns. := RETURN [ PRINCIPAL COMPONENTS ] – output FACTORS, ROTATED_FACTORS [ CLUSTERING [] ] – output CLUSTER_LEVEL, CLUSTER_LEFT, CLUSTER_RIGHT [ CORRELATION ] – output CORRELATIONS [ HIGHLIGHTED ] – output HIGHLIGHTED_DOCUMENT, – HIGHLIGHTED_TERMTYPES TOP { | DEFAULT } [ [as ], … ] – output columns out of

in –

Principal Component keyword when specified, a principal components analysis (factor analysis) is calculated on the correlation matrix of the found documents. Factor Analysis is data reduction method. This is a method of extracting important variables (in form of components) from a large set of variables available in a data set. It extracts low dimensional set of features from a high dimensional data set with a motive to capture as much information as possible. It is always performed on a symmetric correlation or covariance matrix.  The factors will be returned as arrays in the column FACTORS of the result table and the rotated factors will be returned as ARRAYs in the column ROTATED FACTORS. Below graphic in figure 2 shows the transformation of 3-dimensional data to 2-dimensional data (from High to low dimension) using PCA.

Figure 2 PCA Example

Clustering: If [CLUSTERING ] is specified, a hierarchical bottom-up cluster analysis will be performed on the found related documents.

In data mining, hierarchical clustering (also called hierarchical cluster analysis or HCA) is a method of cluster analysis which seeks to build a hierarchy of clusters using bottom up approach.  Set of nested clusters organised as a hierarchical tree. This can be visualized as a dendrogram. Two types of Hierarchical clustering algorithms are Agglomerative and Divisive. Agglomerative starts with data points are individual clusters and at each step, merge the closest pair of clusters till the point just k clusters are formed. Decisive starts with one, all-inclusive cluster, at each step split the cluster till the point k clusters are formed. Figure 3 shows the graphic for HCA.

Figure 3 Nested Cluster and Dendrogram

Find below the list of algorithms available –

‘SINGLE_LINKAGE’: In single linkage, we define the distance between two clusters to be the minimum distance between any single data point in the first cluster and any single data point in the second cluster. This algorithm is sensitive to noise and outliers. Figure 4 shows the graphic for single linkage algorithm.

Figure 4: Single Linkage Nested Cluster and Dendrogram

‘COMPLETE_LINKAGE’: In complete linkage, we define the distance between two clusters to be the maximum distance between any single data point in the first cluster and any single data point in the second cluster. This is more balanced cluster with approximately equal diameter. Figure 5 shows the graphic for complete linkage algorithm.

Figure 5: Complete Linkage Nested Cluster and Dendrogram

‘AVG_DISTANCE_WITHIN’ and ‘AVG_DISTANCE_BETWEEN’: In average linkage, we define the distance between two clusters to be the average distance between data points in the first cluster and data points in the second cluster. Figure 6 shows graphic for Average distance algorithm.

Figure 6: Average Distance Nested Cluster and Dendrogram

‘WARD’: This method looks at cluster analysis as an analysis of variance problem, instead of using distance metrics or measures of association. As per this method, the distance between two clusters, A and B, is how much the sum of squares will increase when we merge them. Figure 7 shows graphic for Ward Algorithm.

Figure 7: Ward Nested Cluster and Dendrogram

The result of the cluster analysis is stored in the columns CLUSTER_LEVEL, CLUSTER_LEFT, and CLUSTER_RIGHT of the result table. Correlation is the association between two variables.

Correlation keyword returns the correlation matrix between the found documents as arrays in the column CORRELATIONS of the result table.  Highlighted keyword returns the document texts with highlighted information.

First Example: In this case, we are pinning it down to one document “Federal_award_id_number = 1304684”. Input is query as part of full text index with document number which is run against the term document matrix/text mining index to fetch top 5 related/similar documents. Score depicts the similarity between the documents, higher the value more similar are the documents. Figure 8a shows the result of function TM_GET_RELATED_DOCUMENTS.

Figure 8a: Result Set of TM_GET_RELATED_DOCUMENTS

First two top ranked documents have same score value ‘1’ means both documents exactly match( Award_abstract column has same content for both the documents). Further list shows that the documents isn’t similar and score value is reducing.

Second Example: This example does statistical analysis.  In this case, we are pinning it down to one document “Federal_award_id_number = 1304684”. Input is query as part of full text index with document number which is run against the term document matrix/text mining index to fetch top 5 related/similar documents. We have principal components, clustering algorithm and correlation matrix. Figure 8b shows the result of function TM_GET_RELATED_DOCUMENTS.

Figure 8b: Result Set of TM_GET_RELATED_DOCUMENTS

TM_GET_RELEVANT_DOCUMENTS

This text mining function returns the top-ranked documents that are relevant to a term. Syntax: TM_GET_RELEVANT_DOCUMENTS ( ) where := TERM [ LANGUAGE ] In this case, specify the term and language to be processed. := SEARCH FROM [ WHERE ] [ WITH TERM TYPE , … ] Specifies the set of reference documents in and . The specified column must be of type text or must have a full-text index. The set of reference documents can be restricted by WHERE or With Term Type. := RETURN [ PRINCIPAL COMPONENTS ] – output FACTORS, ROTATED_FACTORS [ CLUSTERING [] ] – output CLUSTER_LEVEL, CLUSTER_LEFT, – CLUSTER_RIGHT [ CORRELATION ] – output CORRELATIONS [ HIGHLIGHTED ] – output HIGHLIGHTED_DOCUMENT, – HIGHLIGHTED_TERMTYPES TOP { | DEFAULT } [ [as ], … ] – output columns out of –

in

For explanation of above options, refer to previous section. If specified, the options PRINCIPAL COMPONENTS, CLUSTERING, CORRELATION, and [HIGHLIGHTED] must be used in this order. TOP must always be specified as the last option.

Example: Input as “Ocean” which is run against the document matrix/text mining index to fetch top 5 relevant documents. Figure 9 shows the result set of function TM_GET_RELAVANT_DOCUMENTS.

Figure 9: Result Set of TM_GET_RELAVANT_DOCUMENTS

********************************************************************************

In continuation, in next blog [ https://blogs.sap.com/2018/02/18/sap-hana-text-mining-functions-part2/] we have cover Document Classification or Categorization and Term Functions. For details on SAP HANA Text Mining, refer to blog [https://blogs.sap.com/2018/02/16/sap-hana-text-mining/]. http://bit.ly/2DgSJDB #SAP #SAPCloud #AI