There are many ways to integrate Apache NiFi and Apache Spark.
We can call Apache Spark Streaming via S2S (Apache NiFi’s Site to Site) or Kafka. If you want to execute a regular Apache Spark job, you can do that via Apache Livy which is included in HDP 2.6+. This is how Apache Zeppelin integrates with Apache Spark, so it’s secure and a reasonable approach. https://goo.gl/DfGFbg #DataIntegration #ML
