Hybrid Pipelines with Self-Hosted Agents and VM Templates

Configure Hybrid Pipelines and Self-Hosted Agents

Combining Azure and On-Premises Resources

Hybrid pipelines allow you to create a single workflow that uses both Azure-hosted resources and your own on-premises computers or servers. This is done by writing YAML pipeline definitions that can run tasks in both places. This flexibility lets you keep sensitive testing or development tasks on your local network while still using Azure's powerful cloud services for other parts of your build and deployment process. The goal is to maintain a consistent CI/CD operation across these different environments without having to use separate tools.

Using VM Templates for Build Agents

You can use Azure DevTest Labs to create and manage virtual machines (VMs) that act as build or release agents for your pipelines. These VMs are created from preconfigured ARM templates or formulas, which automatically install all the necessary software and tools. This approach gives each development team a private, controlled environment that matches production, allowing for safe testing and parallel builds. The lifecycle of these VMs is managed by the lab, so you can start them when needed for a build and shut them down to save costs.

Setting Up and Securing Self-Hosted Agents

A self-hosted agent is a software program you install on your own VM, whether it's located in Azure or in your on-premises data center. You register these agents to an agent pool in Azure Pipelines, which organizes and controls them. To keep these agents secure, you connect them to Azure using a service connection that is backed by a service principal or a managed identity. This setup allows the agent to authenticate itself and follow your organization's security rules (RBAC) when it needs to access Azure resources like storage or databases.

Building the Pipeline with YAML

To create a hybrid pipeline, you write YAML code that includes specific tasks. These tasks often use command-line tools like the Azure CLI or PowerShell scripts. For example, a pipeline task might create a new VM to act as a self-hosted agent, configure it, and then add it to an agent pool. Another task might use an Azure Resource Manager service connection to deploy an application's infrastructure from an ARM template. By combining these elements, you build a pipeline that can automatically scale its agent capacity and securely handle operations across cloud and on-premises boundaries.