Optimize Parallel Job Configurations
Analyzing Metrics
To improve pipeline speed and manage costs, you must first understand how your resources are being used. This is done by analyzing pipeline throughput and agent utilization metrics. These measurements show you how many jobs are completing over time and whether your build agents are sitting idle or are overloaded. By reviewing this data, you can identify the best level of parallelization—running multiple jobs at once—without wasting money on unused capacity or creating bottlenecks.
Configuring Concurrency Settings
You can control how many jobs run simultaneously by adjusting concurrency settings, either in YAML pipeline files or through the Azure DevOps web interface. This involves setting limits for the minimum and maximum concurrent runs for a workflow. When you enable these settings, you allow more instances of a workflow to run in parallel, which can significantly speed up processing. However, increasing concurrency can sometimes lower other system limits, such as the SplitOn property, so you must consider these trade-offs.
Applying Auto-Scaling and Pool Size Adjustments
To balance performance needs with budget, you can dynamically adjust the number of available build agents. Using auto-scaling allows the system to automatically add or remove agents based on the current workload, ensuring you pay only for what you use. For self-hosted agents, you can scale out by adding more machines to the agent pool to spread the load. In cloud-hosted scenarios, like with an Azure integration runtime, you can scale up by allocating more Data Integration Units (DIUs) to increase the power of a single agent. These adjustments ensure pipelines run quickly when demand is high while reducing costs during quiet periods.