Apache Airflow: The Hands-On Guide

Course Name : Apache Airflow: The Hands-On Guide
Students : 10K+
Duration : 13.5 hrs
Avg Rating : 4.6
Original Price : $119.99
Discount Price : $12.99
Instructor / Provider : Udemy
Course Type : Self Paced Online Course. Lifetime Access
Coupon : Click on ENROLL NOW to apply discount code

What You’ll Learn in the Course

• Airflow Fundamentals: Gain a deep understanding of Apache Airflow, including its core components like the scheduler and the web server.
• Forex Data Pipeline Project: Learn how to build a data pipeline while working with various operators in Airflow, and integrate tools like Slack, Spark, Hadoop, and more.
• Mastering DAGs: Develop expertise in structuring and managing DAGs, including working with timezones, backfill, catchup, unit testing, and organizing DAG folders for clean workflows.
• Scaling Airflow: Learn how to scale Airflow using different executors such as the Local Executor, Celery Executor, and Kubernetes Executor. Discover how to specialize and add workers, and handle node crashes effectively.
• Kubernetes Cluster Setup: Set up and manage Kubernetes clusters locally (3-node cluster with Rancher) and in the cloud (AWS EKS) to run data pipelines using the Kubernetes Executor.
• Advanced Airflow Concepts: Explore practical examples of advanced concepts such as DAG templating, inter-DAG dependencies, SubDAGs, deadlock avoidance, and more.
• Airflow Monitoring: Understand how to monitor Airflow effectively using tools like Elasticsearch and Grafana to ensure optimal performance and pipeline visibility.
• Airflow Security: Learn to secure your Airflow instance with robust authentication, data encryption, and RBAC (Role-Based Access Control). Implement best practices for roles, permissions, and user authentication to maintain security compliance.

Additional Key Highlights

1. Building Production-Grade Pipelines:
   • Hands-on exercises to master coding reliable and scalable data pipelines.
2. Scaling Strategies:
   • Step-by-step guidance on scaling Airflow for high-demand scenarios with Local, Celery, and Kubernetes Executors.
3. Monitoring Best Practices:
   • Implement real-time monitoring with Elasticsearch and Grafana to track pipeline health and performance.
4. Security Implementation:
   • Learn how to set up authentication, encryption, and RBAC to protect sensitive data and secure access to Airflow.
5. Core and Advanced Concepts:
   • Dive into the pros, limitations, and real-world applications of Airflow’s features.

FAQs on Udemy Coupon