Perform advanced streaming data transformations with Apache Spark and Kafka in Azure HDInsight
|
In this module, you learn how to create real-time streaming data analytics pipelines and applications on the cloud by using Azure HDInsight with Apache Kafka and Apache Spark.
When to use Apache Spark and Kafka with HDInsight., Spark Structured Streaming., The architecture of a Kafka and Spark solution., How to provision HDInsight, create a Kafka producer, and stream Kafka data to a Jupyter notebook., How to replicate data to a secondary cluster.
Prerequisites
The following prerequisite should be completed:
Successfully log in to the Azure portal.
Understand the Azure storage options.
Understand the Azure compute options.
Create and configure a HDInsight Cluster in the Azure portal.
User Reviews
Be the first to review “Perform advanced streaming data transformations with Apache Spark and Kafka in Azure HDInsight”
Related Products
Use Delta Lake in Azure Synapse Analytics
Delta Lake is an open source relational storage area for Spark that you can use to implement a data lakehouse architecture in Azure Synapse Analytics.
Introduction to data engineering on Azure
Microsoft Azure provides a comprehensive platform for data engineering; but what is data engineering? Complete this module to find out.
Perform data engineering with Azure Synapse Apache Spark Pools
Apache Spark is a highly scalable distributed processing solution for big data analytics and transformation. You can leverage its power in Azure Synapse Analytics by using Spark pools.
Data Engineering with Microsoft Azure
Explore data engineering concepts on Microsoft Azure’s cloud platform.
Use Spark Notebooks in an Azure Synapse Pipeline
Apache Spark provides data engineers with a scalable, distributed data processing platform, which can be integrated into an Azure Synapse Analytics pipeline.
Data Engineering
Learn how to build, optimize, and maintain data pipelines for data processing and storage.
There are no reviews yet.