Transform data with Spark in Azure Synapse Analytics
|
Data engineers commonly need to transform large volumes of data. Apache Spark pools in Azure Synapse Analytics provide a distributed processing platform that they can use to accomplish this goal.
Use Apache Spark to modify and save dataframes, Partition data files for improved performance and scalability., Transform data with SQL
Prerequisites
Before taking this module, you should be familiar with Apache Spark pools in Azure Synapse Analytics. Consider completing the Analyze data with Apache Spark in Azure Synapse Analytics module first.
User Reviews
Be the first to review “Transform data with Spark in Azure Synapse Analytics”
Related Products
Get started with data engineering on Azure
In most organizations, a data engineer is the primary role responsible for integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions. An Azure data engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints.
Organizational Privacy Engineering
Explore organizational approaches to designing privacy-aware systems and frameworks for data governance.
Data Ingestion with Kafka and Kafka Streaming
Learn how to build real-time data ingestion pipelines using Apache Kafka.
Data Pipelines with Azure
Build scalable data pipelines using Microsoft Azure tools.
Use Azure Synapse serverless SQL pools to transform data in a data lake
By using a serverless SQL pool in Azure Synapse Analytics, you can use the ubiquitous SQL language to transform data in files in a data lake.
Use Apache Spark in Azure Databricks
Azure Databricks is built on Apache Spark and enables data engineers and analysts to run Spark jobs to transform, analyze and visualize data at scale.
There are no reviews yet.