Productionalizing Data Pipelines with Apache Airflow 1
|
This course will teach you how to master production-grade Data Pipelines with ease within Apache Airflow.
Production-grade Data Pipelines are hard to get right. Even when they are done, every update is complex due to its central piece in every organization’s infrastructure. In this course, Productionalizaing Data Pipelines with Apache Airflow 1, you’ll learn to master them using Apache Airflow. First, you’ll explore what Airflow is and how it creates Data Pipelines. Next, you’ll discover how to make your pipelines more resilient and predictable. Finally, you’ll learn how to distribute tasks with Celery and Kubernetes Executors. When you’re finished with this course, you’ll have the skills and knowledge of Apache Airflow needed to make any Data Pipelines production grade.
Author Name: Axel Sirota
Author Description:
Axel Sirota is a Microsoft Certified Trainer with a deep interest in Deep Learning and Machine Learning Operations. He has a Masters degree in Mathematics and after researching in Probability, Statistics and Machine Learning optimisation, he works as an AI and Cloud Consultant as well as being an Author and Instructor at Pluralsight, Develop Intelligence, and O’Reilly Media.
User Reviews
Be the first to review “Productionalizing Data Pipelines with Apache Airflow 1”
Related Products
Data lakes and Lakehouses with Spark and Azure Databricks
Learn to manage and process data using Spark and Azure Databricks platforms.
Explore fundamentals of large-scale analytics
Organizations use analytics platforms to build large scale data analytics solutions that generate insights and drive success. Microsoft provides multiple technologies that you can combine to build a large scale data analytics solution.
Secure data and manage users in Azure Synapse serverless SQL pools
Learn how you can set up security when using Azure Synapse serverless SQL pools
Use Azure Synapse serverless SQL pools to transform data in a data lake
By using a serverless SQL pool in Azure Synapse Analytics, you can use the ubiquitous SQL language to transform data in files in a data lake.
Get started with data engineering on Azure
In most organizations, a data engineer is the primary role responsible for integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions. An Azure data engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints.
Transfer and transform data with Azure Synapse Analytics pipelines
Azure Synapse Analytics enables data integration through the use of pipelines, which you can use to automate and orchestrate data transfer and transformation activities.
There are no reviews yet.