Use Apache Spark in Azure Databricks
|
Azure Databricks is built on Apache Spark and enables data engineers and analysts to run Spark jobs to transform, analyze and visualize data at scale.
Describe key elements of the Apache Spark architecture., Create and configure a Spark cluster., Describe use cases for Spark., Use Spark to process and analyze data stored in files., Use Spark to visualize data.
Prerequisites
Before starting this module, you should have a basic knowledge of Azure Databricks. Consider completing the Explore Azure Databricks module before this one.
User Reviews
Be the first to review “Use Apache Spark in Azure Databricks”
Related Products
Data lakes and Lakehouses with Spark and Azure Databricks
Learn to manage and process data using Spark and Azure Databricks platforms.
Explore fundamentals of large-scale analytics
Organizations use analytics platforms to build large scale data analytics solutions that generate insights and drive success. Microsoft provides multiple technologies that you can combine to build a large scale data analytics solution.
Use Azure Synapse serverless SQL pool to query files in a data lake
With Azure Synapse serverless SQL pool, you can leverage your SQL skills to explore and analyze data in files, without the need to load the data into a relational database.
Transform data with Spark in Azure Synapse Analytics
Data engineers commonly need to transform large volumes of data. Apache Spark pools in Azure Synapse Analytics provide a distributed processing platform that they can use to accomplish this goal.
Get started with data engineering on Azure
In most organizations, a data engineer is the primary role responsible for integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions. An Azure data engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints.
Data Pipelines with Azure
Build scalable data pipelines using Microsoft Azure tools.
There are no reviews yet.