End to End Azure Data Engineering Real Time Project- 3 Hours
End to End Azure Data Engineering Real Time Project- 3 Hours
Published 12/2023
Duration: 2h36m | .MP4 1280x720, 30 fps(r) | AAC, 44100 Hz, 2ch | 1.49 GB
Genre: eLearning | Language: English
Azure Data Engineer Project using the modern Data Tools like ADF, Azure Databricks, Azure Synapse Analytics, ADLS Gen2
What you'll learn
Build real time Azure Data Engineering Project
Data Integration
Build Pipelines
Big Data Transformation
Reporting using Power BI
Requirements
Basic Programming skills
Description
This is a long awaited video of mine- Lets build a complete End to End Azure Data Engineering Project. In this project we are going to create an end to end data platform right from Data Ingestion, Data Transformation, Data Loading and Reporting.
The tools that are covered in this project are,
Azure Data Factory
Azure Data Lake Storage Gen2
Azure Databricks
Azure Synapse Analytics
Azure Key vault
Azure Active Directory (AAD) and
Microsoft Power BI
The use case for this project is building an end to end solution by ingesting the tables from on-premise SQL Server database using Azure Data Factory and then store the data in Azure Data Lake. Then Azure databricks is used to transform the RAW data to the most cleanest form of data and then we are using Azure Synapse Analytics to load the clean data and finally using Microsoft Power BI to integrate with Azure synapse analytics to build an interactive dashboard. Also, we are using Azure Active Directory (AAD) and Azure Key Vault for the monitoring and governance purpose. In this video, I have also covered the complete end to end pipeline testing right from how a new data gets ingested followed by the data transformation until it goes updating the report that we will be creating using the Power BI
Who this course is for:
This course is for absolute beginners who is looking to become an Azure Data Engineer
More Info
What you'll learn
Build real time Azure Data Engineering Project
Data Integration
Build Pipelines
Big Data Transformation
Reporting using Power BI
Requirements
Basic Programming skills
Description
This is a long awaited video of mine- Lets build a complete End to End Azure Data Engineering Project. In this project we are going to create an end to end data platform right from Data Ingestion, Data Transformation, Data Loading and Reporting.
The tools that are covered in this project are,
Azure Data Factory
Azure Data Lake Storage Gen2
Azure Databricks
Azure Synapse Analytics
Azure Key vault
Azure Active Directory (AAD) and
Microsoft Power BI
The use case for this project is building an end to end solution by ingesting the tables from on-premise SQL Server database using Azure Data Factory and then store the data in Azure Data Lake. Then Azure databricks is used to transform the RAW data to the most cleanest form of data and then we are using Azure Synapse Analytics to load the clean data and finally using Microsoft Power BI to integrate with Azure synapse analytics to build an interactive dashboard. Also, we are using Azure Active Directory (AAD) and Azure Key Vault for the monitoring and governance purpose. In this video, I have also covered the complete end to end pipeline testing right from how a new data gets ingested followed by the data transformation until it goes updating the report that we will be creating using the Power BI
Who this course is for:
This course is for absolute beginners who is looking to become an Azure Data Engineer
More Info
