IBM: Apache Spark for Data Engineering and Machine Learning
|
This short course introduces you to the fundamentals of Data Engineering and Machine Learning with Apache Spark, including Spark Structured Streaming, ETL for Machine Learning (ML) Pipelines, and Spark ML. By the end of the course, you will have hands-on experience applying Spark skills to ETL and ML workflows.
About this course
Apache® Spark™ is a fast, flexible, and developer-friendly open-source platform for large-scale SQL, batch processing, stream processing, and machine learning. Users can take advantage of its open-source ecosystem, speed, ease of use, and analytic capabilities to work with Big Data in new ways.
In this short course, you explore concepts and gain hands-on skills to use Spark for data engineering and machine learning applications. You’ll learn about Spark Structured Streaming, including data sources, output modes, operations. Then, explore how Graph theory works and discover how GraphFrames supports Spark DataFrames and popular algorithms.
Organizations can acquire data from structured and unstructured sources and deliver the data to users in formats they can use. Learn how to use Spark for extract, transform and load (ETL) data. Then, you’ll hone your newly acquired skills during your “ETL for Machine Learning Pipelines” lab.
Next, discover why machine learning practitioners prefer Spark. You’ll learn how to create pipelines and quickly implement features for extraction, selections, and transformations on structured data sets. Discover how to perform classification and regression using Spark. You’ll be able to define and identify both supervised and unsupervised learning. Learn about clustering and how to apply the k-mean s clustering algorithm using Spark MLlib. You’ll reinforce your knowledge with focused, hands-on labs and a final project where you will apply Spark to a real-world inspired problem.
Prior to taking this course, please ensure you have foundational Spark knowledge and skills, for example, by first completing the IBM course titled “Big Data, Hadoop and Spark Basics.”
At a Glance:
Institution: IBM
Subject: Computer Science
Level: Intermediate
Prerequisites:
Foundational Apache Spark knowledge and skills.
Associated programs:
Professional Certificate in NoSQL, Big Data and Spark Fundamentals
Professional Certificate in Data Engineering
Language: English
Video Transcript: English
Associated skills:Operations, Extract Transform Load (ETL), Data Engineering, Batch Processing, Unsupervised Learning, Apache Spark, SQL (Programming Language), Machine Learning, Big Data, Stream Processing, Apache Hadoop, Spark Dataframes, Graph Theory, Cluster Analysis
User Reviews
Be the first to review “IBM: Apache Spark for Data Engineering and Machine Learning”
Related Products
An Introduction to Apache Airflow
Learn the basics of Apache Airflow for workflow management, focusing on scheduling, monitoring, and orchestrating data pipelines.
EdinburghX: Data Ethics, AI and Responsible Innovation
Our future is here and it relies on data. Medical robots, smart homes and cities, predictive policing, artificial intelligences – all are fuelled by data and all promise new benefits to society. But will these innovations benefit everyone? Who stands to gain and who is put at risk? How can we ensure that data is part of a just and sustainable world?
AdelaideX: Programming for Data Science
Learn how to apply fundamental programming concepts, computational thinking and data analysis techniques to solve real-world data science problems.
Big Data Systems
Learn about the architecture and management of big data systems.
IBM: NoSQL Database Basics
This course introduces you to the fundamentals of NoSQL, including the four key non-relational database categories. By the end of the course you will have hands-on skills for working with MongoDB, Cassandra and IBM Cloudant NoSQL databases.
AdelaideX: Computational Thinking and Big Data
Learn the core concepts of computational thinking and how to collect, clean and consolidate large-scale datasets.
There are no reviews yet.