The Spark unified analytics engine is one of the most popular frameworks for big data analytics and processing. The GraphFrames package in Apache Spark allows you to represent graphs using a DataFrame-based API. GraphFrames also supports a number of graph algorithms such as Shortest Path, PageRank, Breadth-first search, and connected components. In this course, Executing Graph Algorithms with GraphFrames on Databricks, you will explore how graphs can be used to model entities and relationships in the real world. First, you will learn about the different kinds of graphs such as directed and undirected graphs, weighted and unweighted graphs. Then, you will discover how graphs can be represented using the GraphFrames API in Apache Spark and how you can compute the properties of a graph such as indegree and outdegree of a vertex and perform filtering operations on vertices and edges. Next, you will see how you can perform motif searches using GraphFrames in order to detect structural patterns in the graph. After that, you will learn how to use a domain-specific language for motif finding and run stateless and stateful queries on simple as well as complex real-world graphs. Finally, you will explore the variety of graph algorithms supported by the GraphFrames API including Breadth-first search, Shortest Path, triangle count, connected and strongly connected components, and PageRank. When you are finished with this course, you will have the skills and knowledge of graph algorithms in Spark needed to implement graph algorithms using the GraphFrames API provided by Spark.
Author Name: Janani Ravi
Author Description:
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing … more