Data Engineering for Beginners: Learn SQL, Python & Spark

Course Name : Data Engineering for Beginners: Learn SQL, Python & Spark
Students : 47,041
Duration : 66 hrs
Avg Rating : 4.2
Original Price : $119.99
Discount Price : $9.99
Instructor / Provider : Udemy
Course Type : Self Paced Online Course. Lifetime Access
Coupon : Click on ENROLL NOW to apply discount code

You’ll Learn How to

Environment Setup

• Learn to set up essential tools for Data Engineering, including SQL (Postgres) and Python environments.
• Deploy and configure Hadoop and Spark clusters on Google Cloud Platform (GCP) using Dataproc.
• Use Databricks to leverage Spark Dataframe APIs and Spark SQL for data processing.

SQL Essentials for Data Engineering

• Master SQL fundamentals: creating tables, indexes, and running efficient SQL queries.
• Write queries using WHERE, JOIN, GROUP BY, HAVING, and ORDER BY clauses with practical examples.
• Explore advanced SQL techniques, including cumulative aggregations, ranking, and performance tuning.
• Troubleshoot and debug database-related scenarios to ensure efficient query execution.

Python Essentials for Data Engineering

• Learn Python programming fundamentals, including basic constructs, collections, and debugging.
• Use Pandas for efficient data processing and analysis.
• Work on real-world Python projects:
  • File Format Converter: Convert between file formats like CSV, JSON, and Parquet.
  • Database Loader: Load data into databases efficiently.
• Troubleshoot and optimize Python-based data engineering applications.

Data Engineering with Spark

• Spark DataFrame APIs (PySpark): Master key operations such as select, filter, groupBy, orderBy, and more.
• Spark SQL: Write high-quality SQL queries in Spark using SELECT, WHERE, GROUP BY, and JOIN.
• Build Delta Tables with Spark SQL and perform CRUD operations like INSERT, UPDATE, DELETE, and MERGE.
• Learn advanced Spark SQL techniques, including ranking and partitioning.
• Integrate Spark SQL with PySpark for seamless data engineering workflows.

File Formats and Data Pipelines

• Work with various file formats like Parquet, JSON, and CSV to build robust data pipelines.
• Understand columnar file formats and optimize performance with partitioning.

Apache Spark Application Development

• Understand the complete Spark application lifecycle, including development, debugging, and performance optimization.
• Explore the Spark Catalyst Optimizer for advanced performance tuning.
• Analyze and interpret Explain Plans for Spark SQL queries and PySpark DataFrame operations.

FAQs on Udemy Coupon