From QCon London to Kariera IT Gdańsk – the week-long #GeospatialTuesday

It’s been busy week. That’s why #GeospatialTuesday entry is coming on Friday. Better late than never. The week started with QCon conference in London, where I talked about and demoed geospatial data processing using SAP HANA. At the same time Craig was doing his magic explaining Express edition to the conference participants.

While travelling to London I found that interesting discussion on Reddit (the original post there was deleted for whatever reason) about the size of London comparing to European countries. What a great idea to show the use of SAP HANA and data sets we’ve been using in previous blogs already!

So, the question is what countries in Europe have population smaller than London?

Here is the answer from my SAP HANA Express and the data we already have after previous blogs.

Now let me walk you through how I approached it.

We should have all required data available already:

* GeoNames (which has population data and locations for countries and cities) in schema GEONAME,
* Country shapes in the table “TESTSGEO”.“cntry00”,
* Continents shapes in the table “TESTSGEO”.“continent”

From GeoNames data we need data for the Great Britain (instead of Poland as in the previous blog) and countries (where featurecode is PCLI) only. To get all countries lets download the complete dataset, but then filter and upload only what we need. wget http://download.geonames.org/export/dump/allCountries.zip unzip allCountries.zip cat allCountries.txt | grep PCLI > allPCLI.txt wc -l allPCLI.txt

The output from the last command (counting lines in allPCLI.txt file) was 201 for me at the time of writing.

Let’s load this file into a separate table, so following statements will be more clear. CREATE TABLE “GEONAME”.“PCLIS” LIKE “GEONAME”.“GEONAMES” WITH NO DATA; IMPORT FROM CSV FILE ’/usr/sap/HXE/HDB90/work/geonames/allPCLI.txt’ INTO “GEONAME”.“PCLIS” WITH FIELD DELIMITED BY ’t’ DATE FORMAT ‘YYYY-MM-DD’ ERROR LOG ‘PCLI.err’ NO TYPE CHECK THREADS 8 OPTIONALLY ENCLOSED BY “ COLUMN LIST ("GEONAMESID”, “NAME”, “ASCIINAME”, “ALTERNATENAMES”, “LATITUDE”, “LONGITUDE”, “FEATURECLASS”, “FEATURECODE”, “COUNTRYCODE”, “CC2”, “ADMIN1CODE”, “ADMIN2CODE”, “ADMIN3CODE”, “ADMIN4CODE”, “POPULATION”, “ELEVATION”, “DEM”, “TIMEZONE”, “MODIFICATIONDATE”); UPDATE “GEONAME”.“PCLIS” SET “LOC_4326” = ST_GeomFromText(‘POINT(’ || “LONGITUDE” || ’ ’ || “LATITUDE” || ’)’, 4326), “LOC_3857” = ST_GeomFromText(‘POINT(’ || “LONGITUDE” || ’ ’ || “LATITUDE” || ’)’, 4326).ST_Transform(3857) WHERE “LOC_4326” is null;

Now that all data is there let query it… select ST_UnionAggr(d.“SHAPE”).ST_asGeoJSON() from ( select b.“LOC_4326” from “GEONAME”.“GEONAMES” a join “GEONAME”.“PCLIS” b on a.POPULATION >= b.POPULATION join “TESTSGEO”.“continent” c on c.“SHAPE”.ST_Covers(b.“LOC_4326”) = 1 where a.NAME = ‘London’ and a.FEATURECODE = ‘PPLC’ and c.“CONTINENT” = ‘Europe’ ) c join “TESTSGEO”.“cntry00” d on c.“LOC_4326”.ST_Transform(1000004326).ST_Within(d.“SHAPE”) = 1;

…and use another popular website http://geojsonlint.com/ to visualize the GeoJSON output from the query. That was it.

I am writing this in Gdańsk airport finishing this busy week. Today I presented here at Kariera IT conference. And I am excited as it was the biggest number of participants I had so far at this events series. Next stop will be in Warsaw on March 24th. See you there? http://bit.ly/2pPPoHR #SAP #SAPCloud #AI

ABAP Meets #Blockchain

Yesterday I was reading about Blockchain, its architecture and other technical detail. I being from ABAP background thought of coding a basic blockchain in ABAP.  Please note that intent here is to build a basic blockchain so that its understanding can be increased. Implementation of the blockchain may differ based on the requirements but basic concept remains the same.

The other features such as power of distributed data management and sync, validation of blocks etc. is out of scope of this blog.

Blockchain Basics

As the name suggests Blockchain is basically chain or collection of blocks. Typically a block comprises of a header which has different fields, a hash (cryptic value) of previous block and Transactions Data in hashed form (we will dig into this in our next blog). The first block in the blockchain is known as genesis block. N number of transactions together make a block. These all block are distributed over the network as local copy on each connected node.

Once the maximum number of transactions in a block is reached, the three parts of the block which are header, data(detail of transactions) & previous block hash are  combined and hashed(encrypted ) with a key known as proof of work or nonce.  The correct key or proof of work is determined based on the difficulty level.  For example we say a generated hash is valid only and only if it contains 5 leading zero’s. So this leading 5 zero’s is known as the difficulty level.

The process of determining the right key consumes the major time and is known as mining. Depending on which currency it is, rewards are given accordingly to the one who finds a valid Proof of Work.  The moment someone finds a valid proof of work all the connected nodes are informed and if 50% of the nodes agreed then this new hash is added as a new block to the chain.

What we are planning to build

* A basic prototype of Blockchain, where a report will take input as difficulty level and number of blocks to be generated and then output will be hash and nonce for value for each hash. In the current blog our focus is to highlight how it works. In next blog we will focus on how the transactions are arranged in a block:)

ABAP Implementation of basic Blockchain

* Created a basic report with below mentioned selection screen parameter

* A Type for basic blockchain structure

* Genesis block

* First step is to add genesis block, we are using SHA1 algorithm for encryption.

* Determine Next Block Hash

* An iterative loop to repeat number of blocked needed – 1 since genesis block is already added.
* Inside block loop have another loop to concatenate current and previous hash data along with Nonce value and generate a hash. Validate the hash as per difficulty if valid exit and append the block

* Output Display

What is expected next?

* Deep dive into Block transactions arrangement – Merkel tree
* Implementing Blockchain using BOPF – This will be more like learning BOPF by implementing a blockchain using it
* Adding flavor of CDS to Blockchains with annotations:)
* Finally if time permits a HCP UI5 app to display and run this chain rather than traditional report

Feel free to provide your feedback,open to all ears:). Lets share and learn. http://bit.ly/2pSH1LI #SAP #SAPCloud #AI