Azure Databricks Data Engineer Associate DP-750 Course in Pretoria

Azure Databricks Data Engineer Associate DP-750 Course Outline

Module 1: Explore Azure Databricks

• Introduction
• Get Started with Azure Databricks
• Identify Azure Databricks Workloads
• Understand Key Concepts
• Data Governance Using Unity Catalog and Microsoft Purview
• Exercise - Explore Azure Databricks

Module 2: Understand Azure Databricks Architecture

• Introduction
• Understand Azure Databricks Architecture
• Understand Unity Catalog Managed Storage
• Understand External Storage
• Understand Default Storage (Serverless Compute)

Module 3: Understand Azure Databricks Integrations

• Introduction
• Understand Integration with Microsoft Fabric
• Understand Integration with Power BI
• Understand Integration with VS Code
• Understand Integration with Power Platform
• Understand Integration with Copilot Studio
• Understand Integration with Microsoft Purview
• Understand Integration with Microsoft Foundry

Module 4: Select and Configure Compute in Azure Databricks

• Introduction
• Choose an Appropriate Compute Type
• Configure Compute Performance
• Configure Compute Features
• Install Libraries for Compute
• Configure Compute Access
• Exercise - Select and Configure Compute in Azure Databricks

Module 5: Create and Organize Objects in Unity Catalog

• Introduction
• Apply Naming Conventions
• Create Catalog
• Create Schema
• Create Tables and Views
• Create Volumes
• Implement DDL Operations
• Implement Foreign Catalog
• Configure AI/BI Genie Instructions
• Exercise - Create and Organize Objects in Unity Catalog

Module 6: Secure Unity Catalog Objects

• Introduction
• Understand Query Lifecycle
• Implement Access Control Strategies
• Understand Fine-Grained Access Control
• Implement Row Filtering and Column Masking
• Access Azure Key Vault Secrets
• Authenticate Data Access with Service Principals
• Authenticate Resource Access with Managed Identities
• Exercise - Secure Unity Catalog Objects

Module 7: Govern Unity Catalog Objects

• Introduction
• Create and Preserve Table Definitions
• Configure ABAC with Tags and Policies
• Apply Data Retention Policies
• Set Up and Manage Data Lineage
• Configure Audit Logging
• Design Secure Delta Sharing Strategy
• Exercise - Govern Unity Catalog Objects

Module 8: Design and Implement Data Modeling with Azure Databricks

• Introduction
• Design Ingestion Logic and Data Source Configuration
• Choose a Data Ingestion Tool
• Choose a Data Table Format
• Design and Implement a Data Partitioning Scheme
• Choose a Slowly Changing Dimension (SCD) Type
• Implement a Slowly Changing Dimension (SCD) Type 2
• Design and Implement a Temporal (History) Table to Record Changes over Time
• Choose Granularity on a Column or Table Based on Requirements
• Choose Managed vs. Unmanaged Tables
• Design and Implement a Clustering Strategy
• Exercise - Design and Implement Data Modeling with Azure Databricks

Module 9: Ingest Data into Unity Catalog

• Introduction
• Ingest Data with Lakeflow Connect
• Ingest Data with Notebooks
• Ingest Data with SQL Methods
• Ingest Data with CDC Feed
• Ingest Data with Spark Structured Streaming
• Ingest Data with Auto Loader
• Ingest Data with Lakeflow Spark Declarative Pipelines
• Exercise - Ingest Data into Unity Catalog

Module 10: Cleanse, Transform, and Load Data into Unity Catalog

• Introduction
• Profile Data
• Choose Column Data Types
• Resolve Duplicates and Nulls
• Transform Data with Filters and Aggregations
• Transform Data with Joins and Set Operators
• Transform Data with Denormalization and Pivots
• Load Data with Merge, Insert, and Append
• Exercise - Cleanse, Transform, and Load Data into Unity Catalog

Module 11: Implement and Manage Data Quality Constraints with Azure Databricks

• Introduction
• Implement Validation Checks
• Implement Data Type Checks
• Detect and Manage Schema Drift
• Manage Data Quality with Pipeline Expectations
• Exercise - Implement and Manage Data Quality Constraints with Azure Databricks

Module 12: Design and Implement Data Pipelines with Azure Databricks

• Introduction
• Design Order of Operations for a Pipeline
• Choose Notebook vs. Lakeflow Pipelines
• Design Lakeflow Job Logic
• Design Error Handling in Pipelines and Jobs
• Create Pipeline with Notebook
• Create Pipeline with Lakeflow Spark Declarative Pipelines
• Exercise - Design and Implement Data Pipelines with Azure Databricks

Module 13: Implement Lakeflow Jobs with Azure Databricks

• Introduction
• Create Job Setup and Configuration
• Configure Job Triggers
• Schedule a Job
• Configure Job Alerts
• Configure Automatic Restarts
• Exercise - Implement Lakeflow Jobs with Azure Databricks

Module 14: Implement Development Lifecycle Processes in Azure Databricks

• Introduction
• Apply Git Version Control Best Practices
• Manage Branching and Pull Requests
• Implement Testing Strategy
• Configure and Package DABs
• Deploy Bundle with Databricks CLI
• Exercise - Implement Development Lifecycle Processes in Azure Databricks

Module 15: Monitor, Troubleshoot, and Optimize Workloads in Azure Databricks

• Introduction
• Monitor and Manage Cluster Consumption
• Troubleshoot and Repair Lakeflow Jobs
• Troubleshoot Spark Jobs and Notebooks
• Investigate Caching, Skewing, Spilling, and Shuffle
• Implement Log Streaming with Azure Log Analytics
• Exercise - Monitor, Troubleshoot, and Optimize Workloads in Azure Databricks

Who Should Attend this Azure Databricks Data Engineer Associate DP-750 Course?

This course is designed for professionals looking to build expertise in data engineering using Azure Databricks. It is suitable for individuals aiming to develop skills in data pipelines, processing, and governance. Delegates who will benefit from this training include:

• Data Engineer
• Data Analyst
• Cloud Engineer
• BI Developer
• ETL Developer
• Database Administrator
• Software Engineer
• IT Professional

Prerequisites for this Azure Databricks Data Engineer Associate DP-750 Course

There are no formal prerequisites for this course. However, a basic understanding of data concepts, SQL, and cloud computing will be beneficial.

Azure Databricks Data Engineer Associate DP-750 Course Overview

The Azure Databricks Data Engineer Associate DP-750 Training provides delegates with a comprehensive understanding of data engineering processes using Azure Databricks. It focuses on data ingestion, transformation, governance, and pipeline development.

This training helps delegates build practical skills in managing data workflows, ensuring data quality, and optimising performance. It supports improved efficiency, scalability, and governance in data-driven environments.

This 4-Day course offered by The Knowledge Academy enables delegates to apply data engineering techniques effectively. Delegates gain the confidence to design and manage robust data solutions in real-world scenarios.

Azure Databricks Data Engineer Associate DP-750 Course Objectives

• To understand Azure Databricks architecture and core data engineering concepts
• To design and implement data ingestion processes and ETL pipelines
• To manage and transform data using Unity Catalog and Spark frameworks
• To apply data governance, security, and access control strategies
• To automate workflows and manage data pipeline execution efficiently
• To monitor, troubleshoot, and optimise data processing workloads

Upon completion of this Azure Databricks Data Engineer Associate DP-750 Course, delegates will be able to build and manage data pipelines effectively. They will also enhance their ability to process, govern, and optimise data workflows in Azure environments.

What's Included in this Azure Databricks Data Engineer Associate DP-750 Course?

• World-Class Training Sessions from Experienced Instructors
• Azure Databricks Data Engineer Associate DP-750 Certificate
• Interactive Learning with 24*7 Support
• Digital Delegate Pack