Spark is possibly the most popular engine for big data processing these days. In this course, Building Machine Learning Models in Spark 2, you will learn to build and train Machine Learning (ML) models such as regression, classification, clustering, and recommendation systems on Spark 2.x’s distributed processing environment. This course starts off with an introduction of the 2 ML libraries available in Spark 2; the older spark.mllib library built on top of RDDs and the newer spark.ml library built on top of dataframes. You will get to see the two compared to help you know when to pick one over the other. You will get to see a classification model built using Decision Trees the old way, and see how you can implement the same model on the newer spark.ml library. The course covers many features of Spark 2, including going over a brand new feature in Spark 2, the ML pipelines used to chain your data transformations and ML operations. At the end of this course you will be comfortable using the advanced features that Spark 2 offers for machine learning. You’ll learn to use components such as Transformers, Estimators, and Parameters within your ML pipelines to work with distributed training at scale.
Author Name: Janani Ravi
Author Description:
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing … more

Table of Contents

• Course Overview
  2mins
• Machine Learning Packages: spark.mllib vs. spark.ml
  48mins
• Building Classification and Regression Models in Spark ML
  70mins
• Implementing Clustering and Dimensionality Reduction in Spark ML
  46mins
• Building Recommendation Systems in Spark ML
  39mins