The Databricks Data Lakehouse platform offers a managed environment to train and compare your deep learning models, perform hyperparameter tuning, and productionize and serve your models. In this course, Building Deep Learning Models on Databricks, you will learn to use Bamboolib for no-code data analysis and transformations. First, you will build deep learning models using TensorFlow 2.0 and Keras, and will create a workspace experiment to manage your runs and use autologging to track model parameters, metrics, and artifacts. Next, you will compare multiple runs to find the best-performing model using the MLflow UI. Then, you will see that in order to have support for autologging in MLflow you need to use the PyTorch Lightning framework to design and train your model. You will also register your model with the model registry and use it for batch inference, deploy a Classic MLflow endpoint to serve model predictions, and use the Horovod framework for distributed training of your model. Finally, you will learn how you can use the Hyperopt tool for hyperparameter tuning of your deep learning models, and will run hyperparameter tuning in a distributed fashion on a Spark cluster using the SparkTrials class. When you are finished with this course, you will have the skills and knowledge to build and train deep machine learning models on Databricks using MLflow to manage your machine learning workflow.
Author Name: Janani Ravi
Author Description:
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing … more