Use Spark Notebooks in an Azure Synapse Pipeline
|
Apache Spark provides data engineers with a scalable, distributed data processing platform, which can be integrated into an Azure Synapse Analytics pipeline.
Describe notebook and pipeline integration., Use a Synapse notebook activity in a pipeline., Use parameters with a notebook activity.
Prerequisites
Before starting this module, you should have experience of using Apache Spark in Azure Synapse Analytics. Consider completing the Analyze data with Apache Spark in Azure Synapse Analytics module first:
User Reviews
Be the first to review “Use Spark Notebooks in an Azure Synapse Pipeline”
Related Products
Secure data and manage users in Azure Synapse serverless SQL pools
Learn how you can set up security when using Azure Synapse serverless SQL pools
Transfer and transform data with Azure Synapse Analytics pipelines
Azure Synapse Analytics enables data integration through the use of pipelines, which you can use to automate and orchestrate data transfer and transformation activities.
Perform data engineering with Azure Synapse Apache Spark Pools
Apache Spark is a highly scalable distributed processing solution for big data analytics and transformation. You can leverage its power in Azure Synapse Analytics by using Spark pools.
Explore fundamentals of large-scale analytics
Organizations use analytics platforms to build large scale data analytics solutions that generate insights and drive success. Microsoft provides multiple technologies that you can combine to build a large scale data analytics solution.
Introduction to data engineering on Azure
Microsoft Azure provides a comprehensive platform for data engineering; but what is data engineering? Complete this module to find out.
Create a lake database in Azure Synapse Analytics
Why choose between working with files in a data lake or a relational database schema? With lake databases in Azure Synapse Analytics, you can combine the benefits of both.
There are no reviews yet.