Spark Fundamentals I
|
Dive into Apache Spark basics. Learn how this distributed computing framework handles big data processing efficiently, with an introduction to its key concepts and operations.
At a Glance
Ignite your interest in Spark with an introduction to the core concepts that make this general processor an essential tool set for working with Big Data.
About This Course Learn the fundamentals of
- Learn how it performs at speeds up to 100 times faster than Map Reduce for iterative algorithms or interactive data mining.
- Learn how it provides in-memory cluster computing for lightning fast speed and supports Java, Python, R, and Scala APIs for ease of development.
- Learn how it can handle a wide range of data processing scenarios by combining SQL, streaming and complex analytics together seamlessly in the same application.
- Learn how it runs on top of Hadoop, Mesos, standalone, or in the cloud. It can access diverse data sources such as HDFS, Cassandra, HBase, or S3.
Course Syllabus
-
Module 1 – Introduction to Spark – Getting started
- What is Spark and what is its purpose?
- Components of the Spark unified stack
- Resilient Distributed Dataset (RDD)
- Downloading and installing Spark standalone
- Scala and Python overview
- Launching and using Spark’s Scala and Python shell ©
-
Module 2 – Resilient Distributed Dataset and DataFrames
- Understand how to create parallelized collections and external datasets
- Work with Resilient Distributed Dataset (RDD) operations
- Utilize shared variables and key-value pairs
-
Module 3 – Spark application programming
- Understand the purpose and usage of the SparkContext
- Initialize Spark with the various programming languages
- Describe and run some Spark examples
- Pass functions to Spark
- Create and run a Spark standalone application
- Submit applications to the cluster
-
Module 4 – Introduction to Spark libraries
- Understand and use the various Spark libraries
-
Module 5 – Spark configuration, monitoring and tuning
- Understand components of the Spark cluster
- Configure Spark to modify the Spark properties, environmental variables, or logging properties
- Monitor Spark using the web UIs, metrics, and external instrumentation
- Understand performance tuning considerations
General Information
- This course is self-paced.
- It can be taken at any time.
- It can be audited as many times as you wish.
Recommended skills prior to taking this course
- Basic understanding of Apache Hadoop and Big Data.
- Basic Linux Operating System knowledge.
- Basic understanding of the Scala, Python, R, or Java programming languages.
Requirements
- Have taken the Hadoop 101 course on BDU.
Course Staff
Henry L. Quach
Alan Barnes
User Reviews
Be the first to review “Spark Fundamentals I”
Related Products
Spark
Gain expertise in using Apache Spark for large-scale data processing and analytics.
EdinburghX: Data Ethics, AI and Responsible Innovation
Our future is here and it relies on data. Medical robots, smart homes and cities, predictive policing, artificial intelligences – all are fuelled by data and all promise new benefits to society. But will these innovations benefit everyone? Who stands to gain and who is put at risk? How can we ensure that data is part of a just and sustainable world?
TokyoTechX: Introduction to Computer Science and Programming
The term “Computation” refers to the action performed by a computer. A computation can be a basic operation and it can also be a sophisticated computer simulation requiring a large amount of data and substantial resources. This course aims at introducing learners with no prior knowledge to the basic key concepts of computer science. By following the lectures and exercises of this course, you will gain an understanding of algorithms by programming using the language Ruby.
Big Data Systems
Learn about the architecture and management of big data systems.
AdelaideX: Big Data Analytics
Learn key technologies and techniques, including R and Apache Spark, to analyse large-scale data sets to uncover valuable business information.
IBM: Big Data, Hadoop, and Spark Basics
This course provides foundational big data practitioner knowledge and analytical skills using popular big data tools, including Hadoop and Spark. Learn and practice your big data skills hands-on.
There are no reviews yet.