Run Azure Databricks Notebooks with Azure Data Factory
|
Using pipelines in Azure Data Factory to run notebooks in Azure Databricks enables you to automate data engineering processes at cloud scale.
Describe how Azure Databricks notebooks can be run in a pipeline., Create an Azure Data Factory linked service for Azure Databricks., Use a Notebook activity in a pipeline., Pass parameters to a notebook.
Prerequisites
Before starting this module, you should have a basic knowledge of Azure Databricks. Consider completing the Explore Azure Databricks module before this one.
User Reviews
Be the first to review “Run Azure Databricks Notebooks with Azure Data Factory”
Related Products
Organizational Privacy Engineering
Explore organizational approaches to designing privacy-aware systems and frameworks for data governance.
Data Ingestion with Kafka and Kafka Streaming
Learn how to build real-time data ingestion pipelines using Apache Kafka.
Introduction to data engineering on Azure
Microsoft Azure provides a comprehensive platform for data engineering; but what is data engineering? Complete this module to find out.
Use Spark Notebooks in an Azure Synapse Pipeline
Apache Spark provides data engineers with a scalable, distributed data processing platform, which can be integrated into an Azure Synapse Analytics pipeline.
Use Azure Synapse serverless SQL pools to transform data in a data lake
By using a serverless SQL pool in Azure Synapse Analytics, you can use the ubiquitous SQL language to transform data in files in a data lake.
Data Engineering
Learn how to build, optimize, and maintain data pipelines for data processing and storage.
There are no reviews yet.