Azure Data Factory (ADF) is Azure’s cloud-managed service for ETL and ELT processes. It is similar to SSIS, but in the cloud.
ADF can connect to various data sources, on-premises or in the cloud. If the data you want to access is on-premises, you will need to configure a data management gateway to connect to the on-premises source.
We will use ADF to implement the data workflow orchestration for WTA Insights and copy csv data files from github to ADLS gen2.
If you are new to this blog, please check the Kickoff series first, to learn about the WTA Insights project and the required Azure setup.
Posts in the ADF Series:
1, Provisioning Azure Data Factory
3. Datasets
5. Parameters, variables and loops – part 1
6. Parameters, variables and loops – part 2
7. ADF integration with GitHub
{ DataWhale.ca } 