Data Engineering using Databricks on AWS and Azure

• Data Engineering leveraging Databricks features
• Databricks CLI to manage files, Data Engineering jobs and clusters for Data Engineering Pipelines
• Deploying Data Engineering applications developed using PySpark on job clusters
• Deploying Data Engineering applications developed using PySpark using Notebooks on job clusters
• Perform CRUD Operations leveraging Delta Lake using Spark SQL for Data Engineering Applications or Pipelines
• Perform CRUD Operations leveraging Delta Lake using Pyspark for Data Engineering Applications or Pipelines
• Setting up development environment to develop Data Engineering applications using Databricks
• Building Data Engineering Pipelines using Spark Structured Streaming on Databricks Clusters
• Incremental File Processing using Spark Structured Streaming leveraging Databricks Auto Loader cloudFiles
• Overview of Auto Loader cloudFiles File Discovery Modes – Directory Listing and File Notifications
• Differences between Auto Loader cloudFiles File Discovery Modes – Directory Listing and File Notifications
• Differences between traditional Spark Structured Streaming and leveraging Databricks Auto Loader cloudFiles for incremental file processing.

Who this course is for:

• Beginner or Intermediate Data Engineers who want to learn Databricks for Data Engineering
• Intermediate Application Engineers who want to explore Data Engineering using Databricks
• Data and Analytics Engineers who want to learn Data Engineering using Databricks
• Testers who want to learn Databricks to test Data Engineering applications built using Databricks

Recommended Course

1. Databricks Certified Data Engineer Professional -Preparation
2. Mastering Databricks SQL Warehouse and Spark SQL
3. Python Web Dev Pro: Flask, Django, HTML, CSS & Bootstrap