Spark Fundamentals II
|
Build on your knowledge of Apache Spark with advanced topics. Explore Spark’s powerful APIs for handling complex data processing tasks in distributed environments.
At a Glance
Building on your foundational knowledge of Spark, take this opportunity to move your skills to the next level. With a focus on Spark Resilient Distributed Data Set operations this course exposes you to concepts that are critical to your success in this field.
ABOUT THIS COURSE
- Get in-deptth knowledge on Spark’s architecture and how data is distributed and tasks are parallelized.
- Learn how to optimize your data for joins using Spark’s memory caching.
- Learn how to use the more advanced operations available in the API.
- The lab exercises for this course are performed exclusively on the Cloud and using a Notebook interface.
COURSE SYLLABUS
- Module 1 – Introduction to Notebooks
- Understand how to use Zeppelin in your Spark projects
- Identify the various notebooks you can use with Spark
- Module 2 – Spark RDD Architecture
- Understand how Spark generates RDDs
- Manage partitions to improve RDD performance
- Module 3 – Optimizing Transformations and Actions
- Use advanced Spark RDD operations
- Identify what operations cause shuffling
- Module 4 – Caching and Serialization
- Understand how and when to cache RDDs
- Understand storage levels and their uses
- Module 5 – Develop and Testing
- Understand how to use sbt to build Spark projects
- Understand how to use Eclipse and IntelliJ for Spark development
GENERAL INFORMATION
- This course is self-paced.
- It can be taken at any time.
- It can be audited as many times as you wish.
RECOMMENDED SKILLS PRIOR TO TAKING THIS COURSE
- Basic understanding of Apache Hadoop and Big Data.
- Have taken the Spark Fundamentals I course on BDU.
- Basic understanding of the Scala, Python, R, or Java programming languages.
- Basic Linux Operating System knowledge.
REQUIREMENTS
- None
COURSE STAFF
Henry L. Quach
Henry L. Quach is the Technical Curriculum Developer Lead for Big Data. He has been with IBM for 9 years focusing on education development. Henry likes to dabble in a number of things including being part of the original team that developed and designed the concept for the IBM Open Badges program. He has a Bachelor of Science in Computer Science and a Master of Science in Software Engineering from San Jose State University.
Alan Barnes
Alan Barnes is a Senior IBM Information Management Course Developer / Consultant. He has worked in several companies as a Senior Technical Consultant, Database Team Manager, Application Programmer, Systems Programmer, Business Analyst, DB2 Team Lead and more. His career in IT spans more than 35 years.
User Reviews
Be the first to review “Spark Fundamentals II”
Related Products
CSE 8803 Special Topics: Big Data
Explore advanced topics in big data technologies, architectures, and analytical methods.
Building Scalable Data Pipelines with Kafka
This course brings any Big Data newbie up to speed on one of the most widely used open-source platform to build data pipelines - Apache Kafka.
Data Science in Production: Building Scalable Model Pipelines
Learn how to build scalable model pipelines for data science in production environments, focusing on best practices for deployment and management.
Big Data Fundamentals with PySpark
Learn the fundamentals of working with big data with PySpark.
Deploying a Hadoop Cluster
Gain practical knowledge on how to deploy and manage a Hadoop cluster for big data processing.
IBM: Introduction to Data Science
Learn about the world of data science first-hand from real data scientists.
There are no reviews yet.