Azure DevOps: Multi-Stage Pipelines with YAML

The Classic UI pipelines in Azure DevOps are going away. The future is YAML—specifically Multi-Stage Pipelines. This approach allows you to define your Build (CI) and Deployment (CD) processes in a single, version-controlled file stored alongside your code. This guide implements a complete specific end-to-end pipeline for a .NET Core application deploying to Azure Kubernetes Service (AKS).

Pipeline Architecture

flowchart LR
    DevCommit["Developer Push"] --> BuildStage["Stage: Build"]
    BuildStage --> Test["Unit Tests"]
    Test --> Publish["Publish Artifacts"]
    Publish --> DevDeploy["Stage: Deploy Dev"]
    DevDeploy --> DevGate{Approval?}
    DevGate --> ProdDeploy["Stage: Deploy Prod"]
    
    style BuildStage fill:#E1F5FE,stroke:#0277BD
    style DevDeploy fill:#FFF3E0,stroke:#E65100
    style ProdDeploy fill:#C8E6C9,stroke:#2E7D32

Defining the Infrastructure

trigger:
- main

variables:
  vmImageName: 'ubuntu-latest'
  dockerRegistryServiceConnection: 'acr-connection'
  imageRepository: 'order-api'
  containerRegistry: 'myacr.azurecr.io'
  tag: '$(Build.BuildId)'

stages:
- stage: Build
  displayName: Build and Push
  jobs:
  - job: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: Docker@2
      displayName: Build and Push Docker Image
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: '**/Dockerfile'
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)
          latest

- stage: DeployDev
  displayName: Deploy to Dev
  dependsOn: Build
  jobs:
  - deployment: Deploy
    pool:
      vmImage: $(vmImageName)
    environment: 'dev.aks-namespace'
    strategy:
      runOnce:
        deploy:
          steps:
          - task: KubernetesManifest@0
            inputs:
              action: 'deploy'
              kubernetesServiceConnection: 'aks-dev-connection'
              namespace: 'dev'
              manifests: |
                $(Pipeline.Workspace)/manifests/deployment.yml
                $(Pipeline.Workspace)/manifests/service.yml
              containers: |
                $(containerRegistry)/$(imageRepository):$(tag)

Environments and Approvals

One of the strongest features of YAML pipelines is the environment keyword. This maps to an “Environment” entity in Azure DevOps, where you can configure:

• **Approvals**: Require manual sign-off before the job starts.
• **Checks**: Invoke an Azure Function or query Azure Monitor alerts.
• **Exclusive Lock**: Ensure only one deployment happens at a time.

Using Templates for Reusability

Don’t copy-paste YAML. Use templates to standardize deployment steps.

# templates/deploy-k8s.yml
parameters:
- name: environment
  type: string
- name: namespace
  type: string

jobs:
- deployment: Deploy
  environment: ${{ parameters.environment }}
  strategy:
    runOnce:
      deploy:
        steps:
        - script: echo Deploying to ${{ parameters.namespace }}...
          # deployment steps here

Variable Groups and Key Vault

Never store secrets in YAML. Use Variable Groups backed by Azure Key Vault.

variables:
- group: 'prod-secrets' # Linked to Key Vault

steps:
- script: |
    echo "Using secret connection string..."
    # $(DatabaseConnection) is automatically pulled from KV
  env:
    CONNECTION_STRING: $(DatabaseConnection)

Key Takeaways

• Multi-stage YAML unifies CI and CD.
• Use Environments to enforce manual approvals and gates.
• Templates allow you to write “Deployment as Code” once and reuse.
• Integrate with Azure Key Vault for zero-trust secret management.