Project 12: VM Scale Sets & Advanced Load Balancing
Overview
Deploy highly available, auto-scaling applications using Virtual Machine Scale Sets (VMSS) with Azure Load Balancer. This covers critical compute and networking AZ-104 objectives.
Difficulty: Intermediate to Advanced
Duration: 4-5 hours
Cost: ~$50-100/month (VMs, Load Balancer)
Exam Weight: Compute (20-25%) + Networking (15-20%)
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────────────┐
│ INTERNET │
└─────────────────────────────────────────────────────────────────────────────────┘
│
│ Inbound Traffic
▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ AZURE LOAD BALANCER (Standard SKU) │
│ │
│ ┌───────────────────────────────────────────────────────────────────────────┐ │
│ │ Frontend IP Configuration │ │
│ │ Public IP: 20.x.x.x │ │
│ │ DNS: vmss-demo.eastus.cloudapp.azure.com │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────────────┼───────────────────────────────────────┐ │
│ │ Load Balancing Rules │ │
│ │ │ │ │
│ │ ┌────────────────────┐ ┌──────┴──────┐ ┌────────────────────────┐ │ │
│ │ │ HTTP Rule │ │ HTTPS Rule │ │ Custom Rule │ │ │
│ │ │ Port 80 → 80 │ │ Port 443 │ │ Port 8080 → 8080 │ │ │
│ │ │ Session: None │ │ → 443 │ │ Session: IP │ │ │
│ │ └────────────────────┘ └─────────────┘ └────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────────────┼───────────────────────────────────────┐ │
│ │ Health Probes │ │
│ │ │ │ │
│ │ ┌────────────────────┐ ┌──────┴──────┐ │ │
│ │ │ HTTP Probe │ │ TCP Probe │ Probe Path: /health │ │
│ │ │ Port 80 │ │ Port 443 │ Interval: 5 seconds │ │
│ │ │ Interval: 5s │ │ Interval: │ Threshold: 2 failures │ │
│ │ │ Threshold: 2 │ │ 5s │ │ │
│ │ └────────────────────┘ └─────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────────────┼───────────────────────────────────────┐ │
│ │ Backend Pool │ │
│ │ (VMSS Instances) │ │
│ └───────────────────────────────────┼───────────────────────────────────────┘ │
└──────────────────────────────────────┼───────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ VIRTUAL MACHINE SCALE SET (VMSS) │
│ vmss-webapp │
│ │
│ ┌───────────────────────────────────────────────────────────────────────────┐ │
│ │ Orchestration Mode: Uniform │ │
│ │ OS: Ubuntu 22.04 LTS │ │
│ │ Size: Standard_D2s_v3 │ │
│ │ Upgrade Policy: Automatic │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────────────┐ │
│ │ INSTANCES (Auto-scaling: 2-10) │ │
│ │ │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ ┌──────────┐ │ │
│ │ │ Instance 0 │ │ Instance 1 │ │ Instance 2 │ │ ... │ │ │
│ │ │ Zone 1 │ │ Zone 2 │ │ Zone 3 │ │ up to 10 │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ IP: 10.0.1.4 │ │ IP: 10.0.1.5 │ │ IP: 10.0.1.6 │ │ │ │ │
│ │ │ Nginx │ │ Nginx │ │ Nginx │ │ │ │ │
│ │ │ Web App │ │ Web App │ │ Web App │ │ │ │ │
│ │ └────────────────┘ └────────────────┘ └────────────────┘ └──────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────────────┐ │
│ │ AUTO-SCALING CONFIGURATION │ │
│ │ │ │
│ │ Default Profile: │ │
│ │ ├─ Min instances: 2 │ │
│ │ ├─ Max instances: 10 │ │
│ │ └─ Default: 2 │ │
│ │ │ │
│ │ Scale-out Rule (Add instance): │ │
│ │ └─ When: Average CPU > 70% for 5 minutes │ │
│ │ │ │
│ │ Scale-in Rule (Remove instance): │ │
│ │ └─ When: Average CPU < 30% for 5 minutes │ │
│ │ │ │
│ │ Schedule Profile (Business Hours): │ │
│ │ ├─ Time: Mon-Fri 8:00-18:00 EST │ │
│ │ └─ Min instances: 4 │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────┐
│ NETWORK CONFIGURATION │
│ │
│ ┌───────────────────────────────────────────────────────────────────────────┐ │
│ │ VNet: vnet-vmss (10.0.0.0/16) │ │
│ │ │ │
│ │ ┌────────────────────────────────┐ ┌────────────────────────────────┐ │ │
│ │ │ Subnet: snet-vmss │ │ Subnet: AzureBastionSubnet │ │ │
│ │ │ (10.0.1.0/24) │ │ (10.0.2.0/27) │ │ │
│ │ │ │ │ │ │ │
│ │ │ NSG: nsg-vmss │ │ Azure Bastion │ │ │
│ │ │ - Allow HTTP (80) │ │ (Secure SSH access) │ │ │
│ │ │ - Allow HTTPS (443) │ │ │ │ │
│ │ │ - Allow SSH from Bastion │ │ │ │ │
│ │ └────────────────────────────────┘ └────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────┘
Scaling Timeline Example:
┌──────────────────────────────────────────────────────────────────────────────┐
│ Traffic │ ████████████████████████████ │
│ Level │ ████ ████████████████████████████████ ████ │
│ │ ████ Low ████████ Peak Traffic ████████████ ████ Low │
├───────────┼─────────────────────────────────────────────────────────────────┤
│ Instances│ 2 → 2 → 2 → 4 → 6 → 8 → 10 → 10 → 8 → 6 → 4 → 2 → 2 │
│ │ ▲ ▲ ▲ ▲ │
│ │ │ Scale Out Scale In │ │
│ │ Min (CPU > 70%) (CPU < 30%) Min │
└──────────────────────────────────────────────────────────────────────────────┘VMSS Upgrade Policies
| Policy | Behavior | Use Case |
|---|---|---|
| Manual | No auto-upgrade | Control deployment timing |
| Automatic | All instances at once | Dev/test environments |
| Rolling | Gradual upgrade | Production (recommended) |
What You'll Learn
- Create and configure VMSS
- Set up Azure Load Balancer
- Configure auto-scaling rules
- Implement rolling updates
- Use custom script extensions
- Configure health probes
Phase 1: Create Network Infrastructure
Step 1.1: Create Resource Group
bash
# Set variables
LOCATION="eastus"
RG_NAME="rg-vmss-lab"
VMSS_NAME="vmss-webapp"
# Create resource group
az group create \
--name $RG_NAME \
--location $LOCATION \
--tags Project=VMSS Environment=Lab
echo "Resource group created"Step 1.2: Create Virtual Network
bash
# Create VNet
az network vnet create \
--resource-group $RG_NAME \
--name vnet-vmss \
--address-prefix 10.0.0.0/16 \
--subnet-name snet-vmss \
--subnet-prefix 10.0.1.0/24 \
--location $LOCATION
# Create Azure Bastion subnet
az network vnet subnet create \
--resource-group $RG_NAME \
--vnet-name vnet-vmss \
--name AzureBastionSubnet \
--address-prefix 10.0.2.0/27
echo "VNet created with subnets"Step 1.3: Create Network Security Group
bash
# Create NSG
az network nsg create \
--resource-group $RG_NAME \
--name nsg-vmss \
--location $LOCATION
# Allow HTTP
az network nsg rule create \
--resource-group $RG_NAME \
--nsg-name nsg-vmss \
--name AllowHTTP \
--priority 100 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--destination-port-ranges 80
# Allow HTTPS
az network nsg rule create \
--resource-group $RG_NAME \
--nsg-name nsg-vmss \
--name AllowHTTPS \
--priority 110 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--destination-port-ranges 443
# Allow SSH from Bastion subnet
az network nsg rule create \
--resource-group $RG_NAME \
--nsg-name nsg-vmss \
--name AllowSSHFromBastion \
--priority 200 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--source-address-prefixes 10.0.2.0/27 \
--destination-port-ranges 22
# Associate NSG with subnet
az network vnet subnet update \
--resource-group $RG_NAME \
--vnet-name vnet-vmss \
--name snet-vmss \
--network-security-group nsg-vmss
echo "NSG created and associated"Phase 2: Create Load Balancer
Step 2.1: Create Public IP
bash
# Create public IP for load balancer
az network public-ip create \
--resource-group $RG_NAME \
--name pip-lb-vmss \
--sku Standard \
--allocation-method Static \
--dns-name "vmss-demo-$(date +%s | tail -c 6)" \
--location $LOCATION
# Get the DNS name
LB_DNS=$(az network public-ip show \
--resource-group $RG_NAME \
--name pip-lb-vmss \
--query dnsSettings.fqdn -o tsv)
echo "Load Balancer DNS: $LB_DNS"Step 2.2: Create Load Balancer
bash
# Create Standard Load Balancer
az network lb create \
--resource-group $RG_NAME \
--name lb-vmss \
--sku Standard \
--public-ip-address pip-lb-vmss \
--frontend-ip-name FrontEnd \
--backend-pool-name BackEndPool \
--location $LOCATION
echo "Load Balancer created"Step 2.3: Create Health Probe
bash
# Create HTTP health probe
az network lb probe create \
--resource-group $RG_NAME \
--lb-name lb-vmss \
--name HealthProbe \
--protocol Http \
--port 80 \
--path "/health" \
--interval 5 \
--threshold 2
echo "Health probe created"Step 2.4: Create Load Balancing Rule
bash
# Create load balancing rule for HTTP
az network lb rule create \
--resource-group $RG_NAME \
--lb-name lb-vmss \
--name HTTPRule \
--protocol Tcp \
--frontend-port 80 \
--backend-port 80 \
--frontend-ip-name FrontEnd \
--backend-pool-name BackEndPool \
--probe-name HealthProbe \
--idle-timeout 15 \
--enable-tcp-reset true
echo "Load balancing rule created"Step 2.5: Create Inbound NAT Rules (for SSH)
bash
# Create NAT pool for SSH access to individual instances
az network lb inbound-nat-rule create \
--resource-group $RG_NAME \
--lb-name lb-vmss \
--name SSHNatRule \
--protocol Tcp \
--frontend-port-range-start 50000 \
--frontend-port-range-end 50099 \
--backend-port 22 \
--frontend-ip-name FrontEnd
echo "NAT rules created for SSH access"Phase 3: Create VM Scale Set
Step 3.1: Create Custom Script for Web Server
bash
# Create cloud-init script
cat > cloud-init.yaml << 'EOF'
#cloud-config
package_update: true
packages:
- nginx
- jq
write_files:
- path: /var/www/html/index.html
content: |
<!DOCTYPE html>
<html>
<head>
<title>VMSS Demo</title>
<style>
body { font-family: Arial; margin: 40px; background: #f0f0f0; }
.container { background: white; padding: 20px; border-radius: 10px; box-shadow: 0 2px 5px rgba(0,0,0,0.1); }
.hostname { color: #0078d4; font-size: 24px; }
.info { margin: 10px 0; padding: 10px; background: #f5f5f5; border-radius: 5px; }
</style>
</head>
<body>
<div class="container">
<h1>Azure VM Scale Set Demo</h1>
<p class="hostname">Hostname: <span id="hostname">Loading...</span></p>
<div class="info">
<p><strong>Instance IP:</strong> <span id="ip">Loading...</span></p>
<p><strong>Timestamp:</strong> <span id="time">Loading...</span></p>
</div>
<p>Refresh the page to see load balancing in action!</p>
</div>
<script>
document.getElementById('hostname').textContent = '${HOSTNAME}';
document.getElementById('time').textContent = new Date().toISOString();
fetch('http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2021-02-01&format=text', {headers: {'Metadata': 'true'}})
.then(r => r.text())
.then(ip => document.getElementById('ip').textContent = ip)
.catch(() => document.getElementById('ip').textContent = 'N/A');
</script>
</body>
</html>
- path: /var/www/html/health
content: |
{"status": "healthy", "timestamp": ""}
- path: /etc/nginx/sites-available/default
content: |
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
index index.html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
location /health {
add_header Content-Type application/json;
return 200 '{"status": "healthy", "hostname": "$hostname"}';
}
}
runcmd:
- sed -i "s/\${HOSTNAME}/$(hostname)/g" /var/www/html/index.html
- systemctl restart nginx
- systemctl enable nginx
EOF
echo "Cloud-init script created"Step 3.2: Create VMSS
bash
# Get subnet ID
SUBNET_ID=$(az network vnet subnet show \
--resource-group $RG_NAME \
--vnet-name vnet-vmss \
--name snet-vmss \
--query id -o tsv)
# Get load balancer backend pool ID
LB_BACKEND_ID=$(az network lb address-pool show \
--resource-group $RG_NAME \
--lb-name lb-vmss \
--name BackEndPool \
--query id -o tsv)
# Create VMSS
az vmss create \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--image Ubuntu2204 \
--vm-sku Standard_D2s_v3 \
--instance-count 2 \
--admin-username azureadmin \
--generate-ssh-keys \
--vnet-name vnet-vmss \
--subnet snet-vmss \
--lb lb-vmss \
--backend-pool-name BackEndPool \
--upgrade-policy-mode Automatic \
--custom-data cloud-init.yaml \
--zones 1 2 3 \
--location $LOCATION
echo "VMSS created with 2 instances across 3 zones"Step 3.3: Verify VMSS Instances
bash
# List instances
az vmss list-instances \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--output table
# Get instance IPs
az vmss nic list \
--resource-group $RG_NAME \
--vmss-name $VMSS_NAME \
--query "[].{Instance:virtualMachine.id, IP:ipConfigurations[0].privateIPAddress}" \
--output tablePhase 4: Configure Auto-Scaling
Step 4.1: Create Auto-Scale Settings
bash
# Get VMSS resource ID
VMSS_ID=$(az vmss show \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--query id -o tsv)
# Create autoscale settings
az monitor autoscale create \
--resource-group $RG_NAME \
--resource $VMSS_ID \
--resource-type Microsoft.Compute/virtualMachineScaleSets \
--name "vmss-autoscale" \
--min-count 2 \
--max-count 10 \
--count 2
echo "Autoscale settings created"Step 4.2: Add Scale-Out Rule
bash
# Scale out when CPU > 70%
az monitor autoscale rule create \
--resource-group $RG_NAME \
--autoscale-name "vmss-autoscale" \
--condition "Percentage CPU > 70 avg 5m" \
--scale out 2
echo "Scale-out rule created: Add 2 instances when CPU > 70%"Step 4.3: Add Scale-In Rule
bash
# Scale in when CPU < 30%
az monitor autoscale rule create \
--resource-group $RG_NAME \
--autoscale-name "vmss-autoscale" \
--condition "Percentage CPU < 30 avg 5m" \
--scale in 1
echo "Scale-in rule created: Remove 1 instance when CPU < 30%"Step 4.4: Add Schedule-Based Profile
bash
# Add business hours profile
az monitor autoscale profile create \
--resource-group $RG_NAME \
--autoscale-name "vmss-autoscale" \
--name "business-hours" \
--min-count 4 \
--max-count 10 \
--count 4 \
--timezone "Eastern Standard Time" \
--start "08:00" \
--end "18:00" \
--recurrence week Mon Tue Wed Thu Fri
echo "Business hours profile created"Step 4.5: View Auto-Scale Configuration
bash
# View all autoscale settings
az monitor autoscale show \
--resource-group $RG_NAME \
--name "vmss-autoscale" \
--output jsonPhase 5: Test the Deployment
Step 5.1: Test Load Balancer
bash
# Test the application
echo "Testing load balancer..."
echo "URL: http://$LB_DNS"
# Make multiple requests to see different instances
for i in {1..10}; do
curl -s "http://$LB_DNS/health" | jq -r '.hostname'
done
echo ""
echo "Different hostnames indicate load balancing is working!"Step 5.2: Generate Load for Scaling Test
bash
# Connect to an instance and generate CPU load
# First, get instance ID
INSTANCE_ID=$(az vmss list-instances \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--query "[0].instanceId" -o tsv)
# Run stress test (via run-command)
az vmss run-command invoke \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--instance-id $INSTANCE_ID \
--command-id RunShellScript \
--scripts "sudo apt-get install -y stress && stress --cpu 4 --timeout 300 &"
echo "Stress test started - monitor autoscaling"Step 5.3: Monitor Scaling Events
bash
# Watch instance count
watch -n 10 "az vmss list-instances -g $RG_NAME -n $VMSS_NAME --query 'length(@)' -o tsv"
# View autoscale activity log
az monitor autoscale show \
--resource-group $RG_NAME \
--name "vmss-autoscale" \
--query "profiles[0].rules"Phase 6: Rolling Updates
Step 6.1: Configure Rolling Upgrade Policy
bash
# Update VMSS with rolling upgrade policy
az vmss update \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--set upgradePolicy.mode=Rolling \
--set upgradePolicy.rollingUpgradePolicy.maxBatchInstancePercent=20 \
--set upgradePolicy.rollingUpgradePolicy.maxUnhealthyInstancePercent=20 \
--set upgradePolicy.rollingUpgradePolicy.maxUnhealthyUpgradedInstancePercent=20 \
--set upgradePolicy.rollingUpgradePolicy.pauseTimeBetweenBatches=PT5S
echo "Rolling upgrade policy configured"Step 6.2: Update VMSS Image/Configuration
bash
# Create updated cloud-init
cat > cloud-init-v2.yaml << 'EOF'
#cloud-config
package_update: true
packages:
- nginx
write_files:
- path: /var/www/html/index.html
content: |
<!DOCTYPE html>
<html>
<head><title>VMSS Demo v2</title></head>
<body style="background: #0078d4; color: white; font-family: Arial; padding: 40px;">
<h1>VM Scale Set - Version 2.0</h1>
<p>Hostname: ${HOSTNAME}</p>
<p>This is the updated version!</p>
</body>
</html>
runcmd:
- sed -i "s/\${HOSTNAME}/$(hostname)/g" /var/www/html/index.html
- systemctl restart nginx
EOF
# Trigger rolling update
az vmss extension set \
--resource-group $RG_NAME \
--vmss-name $VMSS_NAME \
--name customScript \
--publisher Microsoft.Azure.Extensions \
--version 2.0 \
--settings '{"script": "'"$(base64 -w0 cloud-init-v2.yaml)"'"}'
echo "Rolling update triggered"Step 6.3: Monitor Rolling Update
bash
# View upgrade status
az vmss get-instance-view \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--query "statuses[?code=='ProvisioningState/succeeded']"
# View rolling upgrade status
az vmss rolling-upgrade get-latest \
--resource-group $RG_NAME \
--name $VMSS_NAMEPhase 7: VMSS Management Operations
Step 7.1: Manual Scaling
bash
# Scale to 5 instances
az vmss scale \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--new-capacity 5
# Verify
az vmss list-instances \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--output tableStep 7.2: Reimage Instances
bash
# Reimage specific instance
INSTANCE_ID=$(az vmss list-instances \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--query "[0].instanceId" -o tsv)
az vmss reimage \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--instance-id $INSTANCE_ID
echo "Instance $INSTANCE_ID reimaged"Step 7.3: Upgrade Specific Instances
bash
# Upgrade specific instances (when using Manual policy)
az vmss update-instances \
--resource-group $RG_NAME \
--name $VMSS_NAME \
--instance-ids "*"
echo "All instances upgraded"Summary Table
| Component | Configuration | Notes |
|---|---|---|
| VMSS | Standard_D2s_v3 | Uniform orchestration |
| Load Balancer | Standard SKU | Required for zones |
| Auto-scaling | 2-10 instances | CPU-based rules |
| Upgrade Policy | Rolling | 20% batch size |
| Zones | 1, 2, 3 | High availability |
Exam Tips
- SKU Matching: Standard LB requires Standard public IP
- Zone Redundancy: VMSS zones must match LB zones
- Health Probes: Required for load balancer to route traffic
- Upgrade Policies: Rolling is safest for production
- Auto-scaling: Metric-based + schedule-based profiles
Cleanup
bash
# Remove cloud-init files
rm -f cloud-init.yaml cloud-init-v2.yaml
# Delete resource group
az group delete --name $RG_NAME --yes --no-wait
echo "Cleanup initiated"Key Takeaways
- VMSS: Automatic scaling and management of identical VMs
- Load Balancer: Distributes traffic across healthy instances
- Auto-scaling: Responds to demand automatically
- Rolling Updates: Zero-downtime deployments
- Zone Redundancy: High availability across datacenters
- Health Probes: Essential for traffic routing