Author: admin

As stated in the last article, database obesity due to numerous intermediate tables and stored procedures is rooted in the closed computational system. If there is an independent computing engine providing computing capability independent of databases, then the latter can lose weight.

With a separate computing engine, the database-generated intermediate data doesn’t have to be stored as data tables; instead, it can be stored in the file system to be further computed by the computing engine. The read-only intermediate data, when stored in a file format, doesn’t need to be rewritten but maintaining its compactness and experiencing a higher compression ratio. And transaction consistency isn’t required for its access. Compared with the database, this simple storage and access mechanism enables much better I/O performance. A file system organizes data in a tree structure. It manages the intermediate data generated by different applications (or modules) by category. This is convenient and attaches the intermediate data to its application (or module) to prevent it from being accessed by other applications (or modules). When a module is changed or offline, the intermediate data it generates can be changed accordingly without worrying about the coupling problem caused by data sharing. Similarly, a stored procedure generating intermediate data can also be moved out of the database to become a part of the application to get rid of coupling problem.

Non-database-generated intermediate tables can also be reduced or eliminated. The extract and transform stages of an ETL operation can be handled outside of the database by a computing engine, and then the clean data will be loaded into the database. The first two stages don’t consume database computing resources and thus intermediate tables are not needed to store data. The database is just responsible for storing the final result.

A computing engine can deal with mixed computations for data presentation involving non-database data sources and database data, making it unnecessary to load external data into the database and thus reducing intermediate tables considerably. That the computing engine sends an ad hoc data retrieval request to the data source to get the most recent data for presentation enables a better real-time capability. But by loading data in and storing it as intermediate tables periodically, the most recent data could be missed. Leaving external data where it is helps exploiting strengths of the non-database data sources. NoSQL databases are good at data searching by key values and handle data of various structures well. A professional data computing engine with a good ability of handling multi-level data, like XML and JSON, beats conventional relational databases in phrasing computing logics.

Apart from the essential computing power, a computing engine intended to relieve database burden must possess good openness and be integration-friendly.

The concept of openness refers to the computing capability independent of any storage system. A system with open computing ability can compute data coming from any data sources, like the file system and enables choosing a suitable storage plan to organize and manage the intermediate data. But a computing system requiring a specific data storage mechanism (say the database) is the same old stuff with a different label. The concept of integrability means the computing procedure is embedded into the application to be a part of it, rather than being a separate process that is shared by multiple applications (modules). Thus application coupling won’t happen.

Measured by the two features, the Hadoop system (including Spark) isn’t suitable to work as an open computing engine though it has some computing power. It possesses a certain degree of openness to compute external data, but the performance is poor and seldom is the ability employed. A Hadoop system is huge and runs as an independent process. It lacks nearly any integrability and can’t be fully embedded into an application.

A true open and integration-friendly computing engine enables the separation of computing ability from storage strategy, making it convenient and flexible to design an application’s structure. With such a computing engine, there’s no need to deploy an additional database or scale-out the database in order to access computing power. It lets the database do the job it is best at, making the most use of the resources. https://goo.gl/ow54ST #DataScience #Cloud

I have written in the past about the difference between market demand and operational capacity – and how difficult it is to determine what exactly is being measured in relation to either.  Has the demand for a product declined, or is the organization simply less capable of satisfying it?  For example, the fact there are no bananas in the grocery store does not mean that there is no demand for bananas; but the absence of revenues from the sale of bananas might be regarded, rather erroneously, as a decline in demand.  This is a straight-forward example; and so I doubt that the dynamics would be confused in real life.  How about a decline in clients served by Stan the salesperson – is this a market or operational issue?  It is a question that is not easily answered.  Yet a data scientist might be expected to assert the meaning of data regularly.

In this blog I will be considering the situation from a slightly different angle:  the health of an organization based on metrics of conformance.  These metrics are “prescriptive” – at least in the way I use the term – meaning that a person is highly graded by their ability to perform in a prescribed manner.  This is an aspect of operational capacity.  I will point out an interesting paradox in a moment.  Metrics of conformance is often conflated with metrics of performance.  I will just give an example to demonstrate the difference.  A metric of performance might be the number of products sold.  A metric of conformance might be the number of clients served.  A person can meet a lot of clients but not necessarily sell any products.  I am not going to build on this example since the possibilities are mind warping.

The paradox is that an organization might have, on one hand, reasonable objectives and, on the other, an equally reasonable means of achieving these objectives; yet one might not actually lead to the other.  The metrics might be designed to promote or reward behaviours (the means) that do not help the organization meet its objectives (the ends).  There is a difficult question not so much when to “rethink” the plan that doesn’t seem to be working but really how to build mechanisms or approaches to alert the organization that rethinking is necessary.  When there is a prescriptive regime, and one seems to be adhering to it, and all of the metrics are designed to point to one’s conformance, and there is a belief that conformance will lead to lollipops, it might be difficult to know when something has gone wrong.  Everything might be going “right” in an analytical sense even when everything is going wrong from a business standpoint.

Business conformance is a problematic obstacle.  Because I want readers to understand, if the path to success is all about conformance, this is nothing impressive.  A similar company can be constructed overseas to produce the product at a fraction of the cost by merely following the same formula – in a setting having lower costs.  Or a competitor can immediately take a slice of the local market merely by using copycat tactics.  Moreover, if the market begins to change as the organization is following a particular regime of behaviours and metrics, the likelihood of adapting quickly seems remote.  When everything in the organization is about sticking to its plan, then that plan is precisely what it will stick to, sometimes even to the bitter end.

This blog is inspired to some extent by the great many companies that refused to change with the times – and their apparent inability to recognize the detachment from the market.  So common is this situation, I actually suggest that every person reading this blog belongs to such a company.  Right there in a boardroom somewhere I can imagine somebody pulling out a cereal box:  “Nutritional content … Store in a dry place … Ah, pour milk, enjoy, and lead a balanced life.  That’s the ticket to a long and healthy life!  It is right here on the box.”  I don’t know how many people check the back of cereal boxes to make corporate decisions.  I’m saying that it is sort of normal to behave in a manner similar to this.  Moreover, data scientists might find themselves enablers in the change-prevention process by not really thinking about the strategic implications of their guidance.

Humans were cloned long before other creatures.  Practically every company is run by clones.  They think alike.  They usually look alike.  The army of suits in “The Matrix” enforcing law over a subservient and blissful human population isn’t all that far-fetched.  It is important to appreciate that in order to maintain a competitive difference, it is necessary to seek revenues that are literally different.  The beginning of this process involves developing metric beyond conformance.  It is necessary to get back to basics and start listening to the market.  Do we serve the paradigm – or do we serve the market?  Let us consider what the market actually is – because I think for many, the market is a bunch of shoppers carrying all sorts of plastic and smart devices to make payment.

The market – is complicated.  The market is data – or at least the source of it.  There are faceless masses hidden in the data.  This data has its own character, flow, and design.  In an effort to engage the market, companies impose over the data by extracting from it those aspects regarded as important.  The initial hurdle – in order to break away from the conformance trap – is to let the market set its own metrics.  It should tell its own story.  We participate in the market’s story not the other way around.  A company has to find its place in the market’s amazing story of which we are probably such a small part.  My approach is to use codified narrative – a means of catching different aspects of a dynamic reality.  Recognition through delineation, I call it.  This “delineation” must occur outside the limited scope of enterprise and more within the social context giving rise to the intersection between clients and companies. https://goo.gl/QKCpLz #DataScience #Cloud

The demand for data scientists is rapidly increasing as companies come up to speed with the fourth industrial revolution. The industry is expanding at an exponential rate meaning the term data scientist is becoming ever more ambiguous.  This means it is time to move away from the buzz words and start showing hiring managers the impact that you have had.
 
It’s time for a new resume.

Moving away from the classic resume style that lists ALL of your responsibilities and ALL of your areas of input allows you to really hone in on why you’d be a great fit for the company you are applying to, by tailoring your experiences and projects. Hiring managers and companies aren’t immune to the ambiguity of the industry, they need to know where your expertise lies and how your mathematical genius can have a direct impact on their team, their business and work stream.

Here’s our guide on how best to translate your experience into impact.
 

Format

Job Title, Company

Location, Date

List your projects, in order of most relevant to the position you’re applying to. E.g. if you’re applying to a reinforcement learning position, list these projects first.

-Project name, duration

-The task or problem you tackled, project duration.

-How you approached it (workflow),

-Your contribution (e.g. individual contributor, lead a team in SCRUM style, etc), the technical details (frameworks/languages)

-Business impact/outcome/success – quantify this where possible, e.g. % lift on sales.

For Example

Machine Learning Engineer, Google

Mountain View, 2014 – now

Hired as a Machine Learning Engineer on the Natural Language Processing team to improve the summarization function within Search.

Technical Skills

It’s incredibly important to include your technical skills, ability and knowledge to carry out specific tasks asked of the job you’re applying for. Make sure you highlight what programming languages you use, coding experience and the knowledge of multiple programming languages will help show how strong of a candidate you are.

Think of your formatting as above when listing these skills, putting emphasis on the most relevant and your most advanced.

 
 

Personalise

It’s important to consider when applying for a job, do not blanket apply to every data science role you come across. Personalise your resume for each type of job you are applying to, not all data scientists are created equally.

Include links: Git Hub for engineers, Google Scholar for Researcher incl. h-Index, number of citations.

Make your resume is relevant to the role you are applying for and condense and reduce any detail that doesn’t directly relate!

 
  https://goo.gl/TCvMJT #DataScience #Cloud