Administering a Databricks workspace involves a wide variety of tasks, but one with far-reaching consequences which is often neglected is the management of clusters and cluster policies. In this course, Administering Clusters and Configuring Policies with Databricks Service, you’ll explore how to effectively manage clusters and policies in Databricks which helps ensure teams run their jobs in a consistent environment while keeping costs down to manageable levels. First, you’ll learn the need for cluster policies, and specifically how these can help control costs while still allowing users to run their jobs efficiently. Next, you’ll explore how rules can be set to restrict access to specific Databricks resources, such as instance pools, clusters, and notebooks. Then, you’ll discover how cluster policies can be framed and applied programmatically. Finally, you’ll learn how to generate a personal access token to interact with Databricks using the REST API, and cover how this can be used to define policies, as well as clusters built by applying them. Once you complete this course, you’ll recognize exactly why cluster policies are required, how these can be defined and rolled out in an organization, and how these can be managed with the Databricks REST API.
Author Name: Kishan Iyer
Author Description:
I have a Masters in Computer Science from Columbia University and have worked previously as a developer and DevOps engineer. I now work at Loonycorn which is a studio for high-quality video content. My interests lie in the broad categories of Big Data, ML and Cloud.

Table of Contents

• Course Overview
  2mins
• Defining Cluster Policies on Azure Databricks
  36mins
• Configuring Access to Cluster Resources
  14mins
• Managing and Using Personal Access Tokens
  20mins