What is Apache NiFi?

  Apache NiFi is an open source software to automate and manage the flow of data between different systems .  It provides a web-based UI for...

CONTINUE READING...

What is an RDD and Why Spark needs it?

   Resilient Distributed Data set( RDD) is the core of Apache Spark. It is the fundamental data structure on top of which all the spark comp...

CONTINUE READING...

Deployment modes and Job submission in Apache Spark

   Spark is a Scheduling Monitoring and Distribution engine, it can also acts as a resource manager for its jobs. When Spark runs job by its...

CONTINUE READING...

Capture bad records while loading csv in spark Dataframe

Loading a csv file and capturing all the bad records is a very common requirement in ETL projects. M ost of the relational database loaders ...

CONTINUE READING...