In “ Designing a metadata-driven ETL framework with Azure ADF: An architectural perspective,” I laid the groundwork for a scalable, metadata-driven ETL framework using Azure Data Factory (ADF).

Both AWS Glue and Azure Data Factory have a user-friendly interface and offer pre-built templates and workflows. However, users who are more familiar with either the AWS or Azure ecosystem may find ...

The need for batch movement of data on a regular time schedule is a requirement for most analytics solutions, and Azure Data Factory (ADF) is the service that can be used to fulfil such a requirement.

In this blog, I will show you how to copy SharePoint Lists using Azure Data Factory. And to make copying multiple lists more dynamically, we created control tables along with parameters.

In this blog post, we’ll walk you through how to leverage ADF pipelines for full and incremental backups of your data lake on Azure.

As supercomputer centers have long known and as hyperscalers and cloud builders eventually learned, the larger the cluster, the greater the chance that one of the many components in the system will ...

The pipeline in the data factory you create in this tutorial uses a Copy Activity to copy data from an Azure blob to an Azure SQL database. The Copy Activity performs the data movement in Azure Data ...

The Azure ML Update Resource activity does not generate any output, but in Azure Data Factory, an output dataset is required to drive the Pipeline schedule, so a dummy/placeholder dataset is used in ...