Azure Pipelines Best Practices

This document outlines best practices for creating maintainable, secure, and efficient Azure Pipelines.

How to Apply This Document

Use this deterministic sequence:

1. Apply Security and Version rules first.
2. Add reliability controls (dependsOn, condition, timeoutInMinutes).
3. Apply performance improvements (Cache@2, shallow checkout, scoped artifacts).
4. Add maintainability improvements (displayName, templates, concise comments).
5. Validate the resulting pipeline with azure-pipelines-validator.

Fallback behavior:

• If a recommended task or feature is unavailable in your environment, keep the closest safe alternative and note the tradeoff.
• If you cannot run automatic validation, use manual checks for secrets, task version pinning, hierarchy, and deployment safety.
• If constraints force intentional deviations, list each deviation and explain risk and mitigation.

Security Best Practices

1. Never Hardcode Secrets

❌ Bad:

variables:
  API_KEY: 'sk-1234567890abcdef'
  PASSWORD: 'MyP@ssw0rd'

✅ Good:

variables:
- group: 'my-secrets'  # From variable group
- name: API_KEY
  value: $(SecretApiKey)  # From pipeline variables marked as secret

2. Pin Image and Task Versions

❌ Bad:

pool:
  vmImage: 'ubuntu-latest'

- task: Docker@2

✅ Good:

pool:
  vmImage: 'ubuntu-22.04'  # Specific version

- task: Docker@2  # Specific major version

3. Use Service Connections

Store credentials in service connections, not in pipeline variables.

- task: Docker@2
  inputs:
    containerRegistry: 'myDockerRegistryServiceConnection'  # Service connection
    command: 'login'

4. Mark Sensitive Variables as Secret

variables:
- name: API_TOKEN
  value: $(SecretToken)

# In Azure DevOps UI, mark variable as secret

5. Limit Permissions

Use the principle of least privilege for service connections and agent pools.

Performance Optimization

1. Use Caching

Cache dependencies to speed up builds.

- task: Cache@2
  displayName: 'Cache npm packages'
  inputs:
    key: 'npm | "$(Agent.OS)" | package-lock.json'
    restoreKeys: |
      npm | "$(Agent.OS)"
    path: $(Pipeline.Workspace)/.npm

- script: npm ci --cache $(Pipeline.Workspace)/.npm
  displayName: 'Install dependencies'

2. Optimize Dependencies with dependsOn and condition

Use explicit dependencies to run jobs in parallel when possible.

stages:
- stage: Build
  jobs:
  - job: BuildFrontend
    steps:
    - script: npm run build:frontend

  - job: BuildBackend
    steps:
    - script: npm run build:backend

- stage: Test
  dependsOn: Build
  jobs:
  - job: TestFrontend
    dependsOn: []  # Can start immediately after Build stage
    steps:
    - script: npm test:frontend

  - job: TestBackend
    dependsOn: []  # Can start immediately after Build stage
    steps:
    - script: npm test:backend

3. Use Shallow Clone

Reduce clone time by limiting git history.

steps:
- checkout: self
  clean: true
  fetchDepth: 1  # Shallow clone

4. Use Artifacts Efficiently

Only publish what's needed and set expiration.

- task: PublishPipelineArtifact@1
  inputs:
    targetPath: '$(Build.ArtifactStagingDirectory)/dist'  # Only dist folder
    artifact: 'webapp'
    publishLocation: 'pipeline'

# Set retention in Azure DevOps project settings

5. Use Matrix for Parallel Execution

Test across multiple configurations in parallel.

strategy:
  matrix:
    node18:
      nodeVersion: '18'
    node20:
      nodeVersion: '20'
    node22:
      nodeVersion: '22'
  maxParallel: 3  # Run 3 at a time

Maintainability

1. Use displayName Everywhere

- stage: Build
  displayName: 'Build Application'
  jobs:
  - job: BuildJob
    displayName: 'Build and Compile'
    steps:
    - script: npm run build
      displayName: 'Build with npm'

2. Organize with Stages

Separate concerns into stages for complex pipelines.

stages:
- stage: Build
  displayName: 'Build Stage'
  jobs: [...]

- stage: Test
  displayName: 'Test Stage'
  dependsOn: Build
  jobs: [...]

- stage: Deploy
  displayName: 'Deploy Stage'
  dependsOn: Test
  jobs: [...]

3. Use Templates for Reusability

Extract common logic into templates.

# templates/npm-build.yml
steps:
- task: NodeTool@0
  inputs:
    versionSpec: $(nodeVersion)

- task: Cache@2
  inputs:
    key: 'npm | "$(Agent.OS)" | package-lock.json'
    path: $(Pipeline.Workspace)/.npm

- script: npm ci --cache $(Pipeline.Workspace)/.npm
- script: npm run build

# azure-pipelines.yml
steps:
- template: templates/npm-build.yml
  parameters:
    nodeVersion: '20'

4. Use Variable Groups

Organize variables in variable groups for different environments.

variables:
- group: 'dev-variables'
- group: 'common-variables'

5. Document Your Pipeline

Add comments to explain complex logic.

# This pipeline builds the frontend and backend separately,
# then runs integration tests before deploying to staging.

stages:
- stage: Build
  # We build frontend and backend in parallel to save time
  jobs:
  - job: BuildFrontend
    # Frontend uses React and requires Node 20
    steps: [...]

Pipeline Structure

1. Naming Conventions

# Stage names: PascalCase
- stage: BuildAndTest

# Job names: PascalCase
- job: BuildApplication

# Step displayNames: Sentence case
- script: echo "test"
  displayName: 'Run integration tests'

# Variables: camelCase or snake_case (be consistent)
variables:
  buildConfiguration: 'Release'
  node_version: '20'

2. Logical Stage Organization

stages:
- stage: Build
  jobs: [...]

- stage: UnitTest
  dependsOn: Build
  jobs: [...]

- stage: IntegrationTest
  dependsOn: Build
  jobs: [...]

- stage: DeployStaging
  dependsOn:
  - UnitTest
  - IntegrationTest
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/develop'))
  jobs: [...]

- stage: DeployProduction
  dependsOn:
  - UnitTest
  - IntegrationTest
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
  jobs: [...]

3. Conditions and Triggers

trigger:
  branches:
    include:
    - main
    - develop
    - release/*
  paths:
    exclude:
    - docs/**
    - README.md

pr:
  branches:
    include:
    - main
  paths:
    exclude:
    - docs/**

Deployment Best Practices

1. Use Deployment Jobs for Environments

- deployment: DeployWeb
  displayName: 'Deploy to Production'
  pool:
    vmImage: 'ubuntu-22.04'
  environment:
    name: production
    resourceName: webapp
  strategy:
    runOnce:
      deploy:
        steps:
        - script: echo "Deploying"

2. Add Manual Approval for Production

Configure approvals in the environment settings in Azure DevOps.

3. Use Deployment Strategies

# For zero-downtime deployments
strategy:
  canary:
    increments: [10, 25, 50, 100]
    preDeploy:
      steps:
      - script: echo "Pre-deploy checks"
    deploy:
      steps:
      - script: echo "Deploy to $(strategy.canary.increment)% of instances"
    postDeploy:
      steps:
      - script: echo "Monitor deployment"

4. Implement Rollback Strategy

strategy:
  runOnce:
    deploy:
      steps:
      - script: ./deploy.sh

    on:
      failure:
        steps:
        - script: ./rollback.sh
          displayName: 'Rollback on failure'

Testing

1. Publish Test Results

- script: npm test -- --coverage --ci --reporters=default --reporters=jest-junit
  displayName: 'Run tests'

- task: PublishTestResults@2
  condition: succeededOrFailed()  # Always publish results
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/junit.xml'
    failTaskOnFailedTests: true

2. Publish Code Coverage

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'Cobertura'
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage/cobertura-coverage.xml'

3. Run Linting and Security Scans

- script: npm run lint
  displayName: 'Run ESLint'

- script: npm audit
  displayName: 'Security audit'
  continueOnError: true  # Don't fail build on audit issues

Error Handling

1. Set Timeouts

jobs:
- job: Build
  timeoutInMinutes: 30  # Prevent hung jobs
  cancelTimeoutInMinutes: 5

2. Use Conditions Appropriately

# Always run cleanup
- script: ./cleanup.sh
  displayName: 'Cleanup'
  condition: always()

# Only on failure
- script: ./send-alert.sh
  displayName: 'Send failure notification'
  condition: failed()

# Only on success
- script: ./deploy.sh
  displayName: 'Deploy'
  condition: succeeded()

3. Use continueOnError for Non-Critical Steps

- script: npm run lint
  displayName: 'Run linter'
  continueOnError: true  # Don't fail the pipeline if linting fails

CI/CD Patterns

1. Multi-Environment Deployment

parameters:
- name: deployToStaging
  type: boolean
  default: true
- name: deployToProduction
  type: boolean
  default: false

stages:
- stage: Build
  jobs: [...]

- stage: DeployStaging
  condition: eq(parameters.deployToStaging, true)
  jobs:
  - deployment: DeployStaging
    environment: staging

- stage: DeployProduction
  condition: and(succeeded(), eq(parameters.deployToProduction, true))
  dependsOn:
  - Build
  - DeployStaging
  jobs:
  - deployment: DeployProduction
    environment: production

2. Feature Branch Builds

trigger:
  branches:
    include:
    - main
    - feature/*

# Only deploy from main
- stage: Deploy
  condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
  jobs: [...]

3. Pull Request Validation

pr:
  branches:
    include:
    - main
  paths:
    include:
    - src/**

stages:
- stage: PRValidation
  jobs:
  - job: BuildAndTest
    steps:
    - script: npm install
    - script: npm run build
    - script: npm test
    - script: npm run lint

Common Anti-Patterns to Avoid

❌ Avoid

1. Using latest tags for images or tasks
2. Hardcoding secrets in pipeline files
3. Not using caching for dependencies
4. Not publishing test results
5. Long-running jobs without timeouts
6. Mixing stages/jobs/steps at root level
7. Using unpinned or outdated task major versions
8. Not using displayName
9. Creating monolithic single-stage pipelines for complex workflows
10. Not using templates for repeated logic

Summary Checklist

Before committing your pipeline:

• All secrets are in variables/service connections, not hardcoded
• All images and tasks are pinned to specific versions
• displayName is used for all stages, jobs, and complex steps
• Caching is implemented for package managers
• Test results and coverage are published
• Timeout values are set for long-running jobs
• Deployment jobs use environments for tracking
• Templates are used for repeated logic
• Conditions are used to control deployment to production
• Pipeline is validated before committing

Done Criteria

Best-practice application is complete when:

• All non-negotiable safeguards are present (no hardcoded secrets, pinned versions, immutable deploy tags).
• Reliability and observability controls are included for each build/deploy path.
• Any deviations from this guide are explicitly documented with rationale.
• Validation evidence is included (validator output or manual fallback checks).

Further Reading

• Azure Pipelines Best Practices - Microsoft Learn
• Azure Pipelines Security
• Pipeline caching
• Pipeline runs