Tag: Azure

Most secure way to access Azure Cloud

What if I told you that you could access your Azure Cloud Resources and tick the following boxes:

  • More secure than a VPN
  • More secure than a Bastion Host and Jumpbox
  • No need for patching and SOC2 compliance reports

If your cloud environment is serverless, why settle for less when it comes to remote access?

As soon as you introduce a jump box, you are required to patch the damn thing and this eats into the Opex budget in a big way.

So what is the solution?

What if you had a docker image with all the tools you need for support that you can spin up into an Azure Container Instance and access on demand, with the filesystem running off Azure Blob Storage?

Well, this is possible, and the good news is that you do not need to build it yourself. Microsoft already offers it.

Welcome to Azure Cloud Shell with VNET Integration!

Cloud Shell with VNET integration leveraging subnet delegation for ACI

The only limitation is that the Storage Account only supports Primary Regions, however Microsoft notified me today that Australia East is now supported.

Microsoft is currently working on more secondary region support, just something to be aware of from a security/data sovereignty perspective.

So the experience is like this.

  1. Log into the Azure Portal
  2. Choose Cloud Shell with advanced options – Select VNET Integration and select the subnet / storage account that we will terraform
  3. Boom – You are in the Azure network and Cloud Shell will have all your common support tools that your require.

Terraform

We will need:

  • Dedicated subnet for CloudShell ACI instances to spin up in.

"csh" = "10.1.12.0/26"

  • Dedicated support subnet for storage account

"sup" = "10.1.13.0/26"

  • A dedicate subnet for the Azure Relay
relay_subnet = "10.1.14.0/26"

# Container instance built in OID for permissions / delegation
# Manually grant in Azure AD
container_instance_oid = "4c1b7058-e8ea-4854-abd2-bbb0abb6cd24"
  • Storage Account for OS filesystem – network rules for Cloudshell
# See https://github.com/Azure/azure-quickstart-templates/blob/master/demos/cloud-shell-vnet-storage/azuredeploy.json
resource "azurerm_storage_account" "cloud_shell" {
  name                         = "st${var.client}${var.env}${var.location_shortname}sup"
  resource_group_name          = azurerm_resource_group.rg.name
  location                     = azurerm_resource_group.rg.location
  allow_blob_public_access     = false
  account_tier                 = "Standard"
  access_tier                  = "Cool"
  account_replication_type     = "LRS"
  tags                         = local.tags
  enable_https_traffic_only    = true
  network_rules {
    bypass                     = ["None"]
    default_action             = "Deny"
    ip_rules                   = [ "101.112.8.233" ]
    virtual_network_subnet_ids = [
      azurerm_subnet.cloud_shell_subnet.id,
      azurerm_subnet.subnet["sup"].id
      ]
  }
}

resource "azurerm_storage_share" "cloud_shell" {
  name                 = "cshsupport"
  storage_account_name = azurerm_storage_account.cloud_shell.name
  quota                = 50
}
  • Then a Cloudshell Subnet leveraging subnet delegation so ACI can control network devices
resource "azurerm_subnet" "cloud_shell_subnet" {
  name                 = "snet-${local.resource_id}-${var.client}-csh"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.network.name
  address_prefixes     = [var.cloud_shell_subnet]

  delegation {
    name = "delegation"

    service_delegation {
      name    = "Microsoft.ContainerInstance/containerGroups"
    }
  }
  service_endpoints = [
    "Microsoft.Storage"
  ]
}
  • Azure Relay Subnet and setup
resource "azurerm_subnet" "relay_subnet" {
  name                                            = "snet-${local.resource_id}-${var.client}-rel"
  resource_group_name                             = azurerm_resource_group.rg.name
  virtual_network_name                            = azurerm_virtual_network.network.name
  address_prefixes                                = [var.relay_subnet]
  enforce_private_link_endpoint_network_policies  = true
  enforce_private_link_service_network_policies   = false
}

resource "azurerm_relay_namespace" "relay_namespace" {
  name                = "rel-${local.resource_id}-${var.client}-csh"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku_name = "Standard"
  tags = local.tags
}

resource "azurerm_network_profile" "cloud_shell_containers" {
  name                = "netpr-${local.resource_id}-${var.client}-csh"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  tags = local.tags
  container_network_interface {
    name = "eth-${azurerm_subnet.cloud_shell_subnet.name}"

    ip_configuration {
      name      = "ipconfig-${azurerm_subnet.cloud_shell_subnet.name}"
      subnet_id = azurerm_subnet.cloud_shell_subnet.id
    }
  }
}

resource "azurerm_private_endpoint" "relay_csh" {
  name                = "prve-${local.resource_id}-${var.client}-csh"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  tags                = local.tags

  subnet_id           = azurerm_subnet.relay_subnet.id
  private_service_connection {
    name                           = "prvsc-${local.resource_id}-${var.client}-csh"
    private_connection_resource_id = azurerm_relay_namespace.relay_namespace.id
    subresource_names              = ["namespace"]
    is_manual_connection           = false
  }
}

resource "azurerm_private_dns_zone" "private_dns_relay" {
  name                = "privatelink.servicebus.windows.net"
  resource_group_name = azurerm_resource_group.rg.name
  tags                = local.tags
}

resource "azurerm_private_dns_a_record" "relay_dns_a_record" {
  name                = "csh"
  zone_name           = azurerm_private_dns_zone.private_dns_relay.name
  resource_group_name = azurerm_resource_group.rg.name
  ttl                 = 3600
  records             = [azurerm_private_endpoint.relay_csh.private_service_connection[0].private_ip_address]
}

resource "azurerm_private_dns_zone_virtual_network_link" "relay_vnet_link" {
  name                  = "dnsvl-${local.resource_id}-${var.client}-csh"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.private_dns_relay.name
  virtual_network_id    = azurerm_virtual_network.network.id
  registration_enabled  = false
}

Now connect to CloudShell

So the above should get you on the right track for the most secure and painless free maintenance Remote Access solution for Azure Cloud!

Goodbye VPN!
Goodbye Bastion Host!
Goodbye Jumpboxes!

CloudShell Vnet

Source: https://docs.microsoft.com/en-us/azure/cloud-shell/private-vnet

Advertisement
Azure Streaming Analytics – Beware – JSON Engine

Azure Streaming Analytics – Beware – JSON Engine

After investigating an issue with Azure Streamiung Analytics, we discovered it cannot deserialise JSON that have the same property names but differ in case e.g.

{
  "Proxy": "abc",
  "proxy": "def"
}

If you send the above payload to a Streaming Analytics Job, it will fail.

Source ‘<unknown_location>’ had 1 occurrences of kind ‘InputDeserializerError.InvalidData’ between processing times ‘2020-03-30T00:19:27.8689879Z’ and ‘2020-03-30T00:19:27.8689879Z’. Could not deserialize the input event(s) from resource ‘Partition: [8], Offset: [1], SequenceNumber: [1]’ as Json. Some possible reasons: 1) Malformed events 2) Input source configured with incorrect serialization format

We opened a ticket with Microsoft. This was the response.

“Hi Romiko,

Thank you for being patience with us. I had further discussion with our ASA PG and here’s our findings.

Findings

ASA unfortunately does not support case sensitive column. We understand it is possible for json documents to add to have two columns that differ only in case and that some libraries support it. However there hasn’t been a compelling use case to support it. We will update the documentation as well. 

We are sorry for the inconvenience. If you have any questions or concerns, please feel free to reach out to me. I will be happy to assist you.”

Indeed other libraries do support this, such as powershell, c#, python etc.

C#

using Newtonsoft.Json;

data = “{‘name’: ‘Ashley’, ‘Name’: ‘Romiko’}”

dynamic message = JsonConvert.DeserializeObject(data);

var text = JsonConvert.SerializeObject(message);

I have built the following tool on github as a workaround for streaming data to elastic. https://github.com/Romiko/EventHub-CheckMalformedEvents

A significant reason why Microsoft should support it – is the Elastic Common Schema. (ECS), a new specification that provides a consistent and customizable way to structure your data in Elasticsearch, facilitating the analysis of data from diverse sources. With ECS, analytics content such as dashboards and machine learning jobs can be applied more broadly, searches can be crafted more narrowly, and field names are easier to remember.

When introducing a new schema, there is always dealing with existing/custom data. Elastic have an ingenious way to solve this. All fields in ECS are lower case. So your existing data can be guarnteed to not conflict if you use an UpperCase.

Let us reference Elastic’s advice

https://www.elastic.co/guide/en/ecs/current/ecs-custom-fields-in-ecs.html

image.png

Elastic, who deal with Big Data all the time recommend using Proxy vs proxy to ensure migrations to ECS is a vaiable/conflic free solution.

Conclusion

If you are migrating huge amounts of data to Elastic Common Schema (ECS), Consider if Azure Streaming Analytics is a good fit due to the JSON limits.

You can also vote to fix this issue here and improve Microsoft’s product offering πŸ™‚

https://feedback.azure.com/forums/270577-stream-analytics/suggestions/40122079-azure-stream-analytics-to-be-able-to-handle-case-s

Automate the Deployment of Azure Kubernetes Services Cluster + Application Gateway Ingress Controller

This post will demonstrate how to deploy a AKS cluster using Advanced Networking. We will then deploy an Application Gateway Ingress Controller. Essentially this will install a dedicated ingress POD that fully manages the Application gateway.

This means all entries in the Application gateway are 100% managed by AKS. If you manually add an entry to the AG, it will be removed by AKS Ingress Controller.

Overview

Considerations

  • Decided to dedicate an entire /16 IP range to the AKS cluster for simplicity e.g. 10.69.0.0/16.
  • Leverage AKS with Advanced Networking (CNI).
  • CNI provided the use of Application Gateway with WAFv2.
  • SSL offloading is configured. The actual private key (PEM – Base64 encoded) is stored in the default namespace in AKS. Whenever you deploy a new application, just –export (Deprecated) the key to the new namespace. The AG Ingress Controller will automatically be configured with the SSL certificate.
  • We will apply RBAC rules so AKS can manage the application gateway and VMSS scaleset.
  • RBAC to access container registry.

By using an Application Gateway, we can leverage additional benefits such as Web Application Firewall (V2), OWASP 3.0 firewall detection/prevention rules. Microsoft have totally refactors the AG WAF2 technology stack. It is much faster to provision and can deal with much larger amounts of traffic now.

By combining Load Balancing with WAF, we get the best of both worlds. If you have heavy traffic, it might be good to first do a performance test before making a final decision on AG + AKS stack.

Environment Setup + Tools

We are using AKS VMSS preview feature. Azure Virtual Machine Scale Sets have been around for a long time, and are in fact used by Microsoft Service Fabric. It makes total sense that this auto-scaling architecture is leveraged by AKS.

Due to the preview status of Container Services and VMSS+AKS, we will choose Azure CLI.

You can use Ubuntu Windows Shell or a Linux Ubuntu Shell.

Run the following code to setup your bash environment.

#!/bin/bash
echo "Updating system..."
sudo apt-get update
sudo apt-get upgrade

echo "Installing AzureCLI"
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

echo "Installing helm for AKS Admin"
curl -LO https://git.io/get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh
helm init --service-account tiller --history-max 200

Helm is a client side tool to provide configuration settings to AKS. Tiller is a server side setting that runs on AKS that applies configuration settings that are applied from a helm client.

Create a config folder with 2 files.

Replace THECERTIFICATECHAIN with the contents of your base64 encoded .cer certificate chain. The script will replace <THEPRIVATEKEY> when you paste your private key. Future namespace or apps, will be able to find this in the default namespace. Thus a 1 time operations

e.g. (kubectl get secret rangerrom-tls –namespace=default ….).

apiVersion: v1
kind: Secret
metadata:
  name: rangerrom-tls
type: kubernetes.io/tls
data:
  tls.crt: THECERTIFICATECHAIN
  tls.key: THEPRIVATEKEY

rangerrom-tls.yml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

tiller-rbac.yml

Azure Prerequisites

Ensure you have
* VNET in a resounce group – ${env}-network-rg
* Subnet with a name ${env}-aks-cluster-subnet matching the IP rules

Install AKS into existing VNET

#!/bin/bash
aksversion='1.13.7'
while ! [[ "$env" =~ ^(sb|dv|ut|pd)$ ]] 
do
  echo "Please specifiy environment [sb, dv,ut,pd]?"
  read -r env
done 

case $env in

  dv)
    servicecidr="10.66.64.0/18"
    dnsserver="10.66.64.10"
    az account set --subscription 'RangerRom DEV'
    subscriptionid=$(az account show --subscription 'RangerRom DEV' --query id | sed  's/\"//g')
    ;;

  sb)
    servicecidr="10.69.64.0/18"
    dnsserver="10.69.64.10"
    az account set --subscription 'RangerRom SANDBOX'
    subscriptionid=$(az account show --subscription 'RangerRom SANDBOX' --query id | sed  's/\"//g')
    ;;

  ut)
    servicecidr="10.70.64.0/18"
    dnsserver="10.70.64.10"
    az account set --subscription 'RangerRom TEST'
    subscriptionid=$(az account show --subscription 'RangerRom TEST' --query id | sed  's/\"//g')
    ;;

  pd)
    servicecidr="10.68.64.0/18"
    dnsserver="10.68.64.10"
    az account set --subscription 'RangerRom PROD'
    subscriptionid=$(az account show --subscription 'RangerRom PROD' --query id | sed  's/\"//g')
    ;;  
  *)
    echo "environment not found"
    exit
    ;;
esac

env="rrau${env}"
location="australiaeast"

az group create --location $location --name "${env}-aks-rg"
sleep 5

az feature register -n VMSSPreview --namespace Microsoft.ContainerService
az provider register -n Microsoft.ContainerService

az aks create \
    --resource-group "${env}-aks-rg" \
    --name "${env}-aks-cluster" \
    --enable-vmss \
    --node-count 2 \
    --kubernetes-version $aksversion \
    --generate-ssh-keys \
    --network-plugin azure \
    --service-cidr $servicecidr \
    --dns-service-ip $dnsserver \
    --vnet-subnet-id "/subscriptions/${subscriptionid}/resourceGroups/${env}-network-rg/providers/Microsoft.Network/virtualNetworks/${env}-network/subnets/${env}-aks-cluster-subnet"



clusterprincipalid=$(az ad sp list --display-name ${env}-aks-cluster --query [0].objectId)
resourceGroupid=$(az group show --name ${env}-network-rg --query 'id')
echo "Configuring cluster to owner ${resourceGroupid}"
cmd="az role assignment create --role Contributor --assignee $clusterprincipalid --scope $resourceGroupid"
eval $cmd


echo "Configuring AKS Cluster with Tiller"
az aks get-credentials --resource-group "${env}-aks-rg" --name "${env}-aks-cluster" --overwrite-existing
kubectl apply -f ./config/tiller-rbac.yml
helm init --service-account tiller

while ! [[ ${#privatekey} -gt 2000 ]]
do
  echo "Please provide TLS Private Key - BASE64 Encoded PEM?"
  read -r privatekey
done

kubectl create namespace scpi-${env}
cat ./config/rangerrom-tls.yml  | sed "s/THEPRIVATEKEY/$privatekey/" > temptls.yml
kubectl apply -f temptls.yml -n default
#The Flag --export is going to be deprecated - Below is workaround.
kubectl get secret rangerrom-tls --namespace=default -o yaml | \
   sed '/^.*creationTimestamp:/d' |\
   sed '/^.*namespace:/d' |\
   sed '/^.*resourceVersion:/d' |\
   sed '/^.*selfLink:/d' |\
   sed '/^.*uid:/d' |\
   kubectl apply --namespace=scpi-${env} -f -

rm -f ./temptls.yml
privatekey=""

echo "Setup container registry permissions"
az acr create -n "${env}containerregistry" -g "${env}-common-rg" --sku Premium
containerid=$(az acr show -n ${env}containerregistry --query id)
principalidaks=$(az ad sp list --all --query "([?contains(to_string(displayName),'"${env}-aks-cluster"')].objectId)[0]")
cmd1="az role assignment create --role acrpull --assignee $principalidaks --scope $containerid"
eval $cmd1

echo "Setup container registry permissions - Centralised"
containerid=$(az acr show --subscription 'RangerRom PROD' -n rraupdcontainerregistry --query id)
cmd2="az role assignment create --role acrpull --assignee $principalidaks --scope $containerid"
eval $cmd2

Install and Configure AKS to control the Ingress

#!/bin/bash
while ! [[ "$env" =~ ^(sb|dv|ut|pd)$ ]] 
do
  echo "Ensure you have owner permissions on the subscription before you continue."
  echo "Please specifiy environment [sb, dv,ut,pd]?"
  read -r env
done 

case $env in

  dv)
    az account set --subscription 'RangerRom DEV'
    ;;

  sb)
    az account set --subscription 'RangerRom SANDBOX'
    ;;

  ut)
    az account set --subscription 'RangerRom TEST'
    ;;

  pd)
    az account set --subscription 'RangerRom PROD'
    ;;  
  *)
    echo "Invalid Environment"
    exit
    ;;
esac

env="rrau${env}"

ipAddressName="${env}-aks-application-gateway-ip"
resourcegroup="MC_${env}-aks-rg_${env}-aks-cluster_australiaeast"
gatewayname="${env}-aks-application-gateway"
location="australiaeast"
vnet="${env}-network"
subnet="${env}-aks-application-gateway-subnet"

az network public-ip create \
  --resource-group $resourcegroup \
  --name $ipAddressName \
  --allocation-method Static \
  --sku Standard

sleep 20

subnetid=$(az network vnet subnet show -g "${env}-network-rg" -n "${env}-aks-application-gateway-subnet" --vnet-name ${vnet} --query id)
cmd="az network application-gateway create --name $gatewayname \
                                      --resource-group $resourcegroup \
                                      --capacity 2  \
                                      --sku "WAF_v2" \
                                      --subnet $subnetid \
                                      --http-settings-cookie-based-affinity Disabled \
                                      --location $location \
                                      --frontend-port 80 \
                                      --public-ip-address $ipAddressName"
eval $cmd
az network application-gateway waf-config set -g $resourcegroup --gateway-name $gatewayname \
                            --enabled true --firewall-mode Detection --rule-set-version 3.0

#Setup AAD POD Identity to manage application gateway
az identity create -g $resourcegroup -n "${env}-aks-aad_pod_identity"
sleep 20

principalid=$(az identity show -g $resourcegroup -n "${env}-aks-aad_pod_identity" --query 'principalId')
appgatewayid=$(az network application-gateway show -g $resourcegroup -n $gatewayname --query 'id')

echo "Assign Role so AKS can manage the Application Gateway"

echo "Configuring Create Role for identity - $principalid - for gateway"
cmd="az role assignment create --role Contributor --assignee $principalid --scope $appgatewayid"
eval $cmd

resourceGroupid=$(az group show --name $resourcegroup --query 'id')

echo "Configuring Read Role for identity - $principalid - for gateway resourcegroup"
cmd="az role assignment create --role Reader --assignee $principalid --scope $resourceGroupid"
eval $cmd

az identity show -g $resourcegroup -n "${env}-aks-aad_pod_identity"
echo "Please use the azure identity details above to configure AKS via Help for the AG Ingress Controller"
echo "Careful with copy and paste. Hidden characters can affect the values!"


echo "Ingress Controller for Azure Application Gateway"
az aks get-credentials --resource-group "${env}-aks-rg" --name "${env}-aks-cluster"
kubectl create -f https://raw.githubusercontent.com/Azure/aad-pod-identity/master/deploy/infra/deployment-rbac.yaml
helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/
helm repo update

subscriptionid=$(az account show --query id | sed  's/\"//g')
appGatewayResourceId=$(az network application-gateway show -g $resourcegroup -n $gatewayname --query resourceGroup  | sed  's/\"//g')
identityClientid=$(az identity show -g $resourcegroup -n "${env}-aks-aad_pod_identity" --query clientId  | sed  's/\"//g')
aksfqdn=$(az aks show --resource-group "${env}-aks-rg" --name "${env}-aks-cluster" --query fqdn  | sed  's/\"//g')

cmd="helm upgrade ingress-azure application-gateway-kubernetes-ingress/ingress-azure \
     --install \
     --namespace default \
     --debug \
     --set appgw.name=$gatewayname \
     --set appgw.resourceGroup=$appGatewayResourceId \
     --set appgw.subscriptionId=$subscriptionid \
     --set appgw.shared=false \
     --set armAuth.type=aadPodIdentity \
     --set armAuth.identityResourceID=/subscriptions/$subscriptionid/resourcegroups/$appGatewayResourceId/providers/Microsoft.ManagedIdentity/userAssignedIdentities/$env-aks-aad_pod_identity \
     --set armAuth.identityClientID=$identityClientid \
     --set rbac.enabled=true \
     --set verbosityLevel=3 \
     --set aksClusterConfiguration.apiServerAddress=$aksfqdn"
eval $cmd
kubectl get pods


Conclusion

This post should provide a guide post to setup your infrastructure as code. By leveraging a rock solid naming convention, you can leverage fully automated scripts to deploy your environments. The above scripts for AKS and AG are also idempotent, so they can be run on a scheduled basis e.g. Azure Devops.

Automate #Azure Blob Snapshot backups with @Cerebrata

Hi,

Leveraging the cerebrata cmdlets for Azure, we can easily backup our blob containers via snapshot, this will prove useful for Page Blobs that are Random Access i.e. VHD’s on Cloud Drive

Here is how Purging Snapshots works

#requires -version 2.0
param (
	[parameter(Mandatory=$true)] [string]$AzureAccountName,
	[parameter(Mandatory=$true)] [string]$AzureAccountKey,
	[parameter(Mandatory=$true)] [array]$BlobContainers
)

$ErrorActionPreference = "Stop"

if ((Get-PSSnapin -Registered -Name AzureManagementCmdletsSnapIn -ErrorAction SilentlyContinue) -eq $null)
{
	throw "AzureManagementCmdletsSnapIn missing. Install them from Https://www.cerebrata.com/Products/AzureManagementCmdlets/Download.aspx"
}

Add-PSSnapin AzureManagementCmdletsSnapIn -ErrorAction SilentlyContinue

function SnapShotBlobContainer 
{
	param ( $containers, $blobContainerName )
	Write-Host "Starting snapshot $blobContainerName"

	$container = $containers | Where-Object { $_.BlobContainerName -eq $blobContainerName }

	if ($container -eq $null)
	{
		Write-Host  "Container $blobContainerName doesn't exist, skipping snapshot"
	}
	else
	{
        Write-Host  "Found blob container $blobContainerName"
Checkpoint-BlobContainer -Name $container.BlobContainerName -SaveSnapshotInformation -AccountName $AzureAccountName -AccountKey $AzureAccountKey
	Write-Host  "Snapshot complete for $blobContainerName"
	}
}

$containers = Get-BlobContainer -AccountName $AzureAccountName -AccountKey $AzureAccountKey
foreach($container in $BlobContainers)
{
	SnapShotBlobContainer $containers $container
}

Then just call the script with the params. remember an array of items is parsed in like this:

-BlobContainers:@(‘container1’, ‘contaner2’) -AzureAccountName romikoTown -AzureAccountKey blahblahblahblahblehblooblowblab==

Windows #Azure configuration transformations

Hi,

I needed to update the transformation today to support different csdef files for UAT/Test etc. I found myself forgetting the process, so i thought it would be a good idea to log the entries needed.

What I wanted was a way to disable New Relic in our performance environment, since we do not have a license key.

Since we use the Azure Tasks to run batch jobs before the worker role starts, it made sense that I create a TASK Environment variable that my batch script can check and see if it should install New relic, e.g.

if "%MYSTORYENV%" == "PERF" goto :EOF

So, in the above, my batch file startup.cmd will skip installing new relic if the environment variable is PERF. However we need to set this value in the csdef file.

So we go to the BASE servicedefinition.csdef file and have this entry for it.

<sd:Startup>
      <sd:Task commandLine="Startup.cmd" executionContext="elevated" taskType="background">
        <sd:Environment>
          <sd:Variable name="EMULATED">
            <sd:RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated" />
          </sd:Variable>
          <sd:Variable name="MYSTORYENV" value="DEV" />
        </sd:Environment>
      </sd:Task>
    </sd:Startup>

Notice, that I have qualified all my csdef entries, this is important for transformations to occur (sd:)

Ok, the next step is that we create a transformation file

https://gist.github.com/1777060

Now, that we have this transform, we will need to edit the CSPROJ file. Please see below the parts added

Item Groups

<ItemGroup>
<ServiceDefinition Include="ServiceDefinition.csdef" />
<ServiceConfiguration Include="ServiceConfiguration.cscfg" />
</ItemGroup>
<ItemGroup>
<EnvironmentDefinition Include="ServiceDefinition.uat.csdef">
<BaseConfiguration>ServiceDefinition.csdef</BaseConfiguration>
</EnvironmentDefinition>
<EnvironmentDefinition Include="ServiceDefinition.perf.csdef">
<BaseConfiguration>ServiceDefinition.csdef</BaseConfiguration>
</EnvironmentDefinition>
<EnvironmentConfiguration Include="ServiceConfiguration.uat.cscfg">
<BaseConfiguration>ServiceConfiguration.cscfg</BaseConfiguration>
</EnvironmentConfiguration>
<EnvironmentConfiguration Include="ServiceConfiguration.perf.cscfg">
<BaseConfiguration>ServiceConfiguration.cscfg</BaseConfiguration>
</EnvironmentConfiguration>
<None Include="@(EnvironmentConfiguration)" />
<None Include="@(EnvironmentDefinition)" />
</ItemGroup>

Notice I have the include at the bottom, so I can see these in Visual Studio. I also have transformations for cscfg files, hence the reason why you see them here Smile

Targets Validation

<Target Name="ValidateServiceFiles"
		Inputs="@(EnvironmentConfiguration);@(EnvironmentConfiguration->'%(BaseConfiguration)');@(EnvironmentDefinition);@(EnvironmentDefinition->'%(BaseConfiguration)')"
		Outputs="@(EnvironmentConfiguration->'%(Identity).transformed.cscfg');@(EnvironmentDefinition->'%(Identity).transformed.csdef')">

	<Message Text="ValidateServiceFiles: Transforming %(EnvironmentConfiguration.BaseConfiguration) to %(EnvironmentConfiguration.Identity).tmp via %(EnvironmentConfiguration.Identity)" />
	<TransformXml Source="%(EnvironmentConfiguration.BaseConfiguration)" Transform="%(EnvironmentConfiguration.Identity)" Destination="%(EnvironmentConfiguration.Identity).tmp" />
	<Message Text="ValidateServiceFiles: Transformation complete; starting validation" />

	<Message Text="ValidateServiceFiles: Transforming %(EnvironmentDefinition.BaseConfiguration) to %(EnvironmentDefinition.Identity).tmp via %(EnvironmentDefinition.Identity)" />
	<TransformXml Source="%(EnvironmentDefinition.BaseConfiguration)" Transform="%(EnvironmentDefinition.Identity)" Destination="%(EnvironmentDefinition.Identity).tmp" />
	<Message Text="ValidateServiceFiles: Transformation complete; starting validation" />

	<ValidateServiceFiles ServiceDefinitionFile="@(ServiceDefinition)" ServiceConfigurationFile="%(EnvironmentConfiguration.Identity).tmp" />
	<ValidateServiceFiles ServiceDefinitionFile="%(EnvironmentDefinition.Identity).tmp" ServiceConfigurationFile="@(ServiceConfiguration)" />
	<Message Text="ValidateServiceFiles: Validation complete; renaming temporary file" />

	<Move SourceFiles="%(EnvironmentConfiguration.Identity).tmp" DestinationFiles="%(EnvironmentConfiguration.Identity).transformed.cscfg" />
	<Move SourceFiles="%(EnvironmentDefinition.Identity).tmp" DestinationFiles="%(EnvironmentDefinition.Identity).transformed.csdef" />
</Target>

Notice above I have them for BOTH CSCFG and CSDEF files!

Move transforms to the app.publish folder for azure packaging

<Target Name="MoveTransformedEnvironmentConfigurationXml" AfterTargets="AfterPackageComputeService" Inputs="@(EnvironmentConfiguration->'%(Identity).transformed.cscfg')" Outputs="@(EnvironmentConfiguration->'$(OutDir)app.publish\%(filename).cscfg')">
<Move SourceFiles="@(EnvironmentConfiguration->'%(Identity).transformed.cscfg')" DestinationFiles="@(EnvironmentConfiguration->'$(OutDir)app.publish\%(filename).cscfg')" />
<Move SourceFiles="@(EnvironmentDefinition->'%(Identity).transformed.csdef')" DestinationFiles="@(EnvironmentDefinition->'$(OutDir)app.publish\%(filename).csdef')" />
</Target>

Summary

So there you have it, you will now have csdef and cscfg files for different environments.

Windows #Azure–Pre Role Startup Tasks

Hi,

Imagine you need to boot up a web role in the cloud, but before the global.asax events kick in or even lower, before the WebRoleEntryPoint events kick in, you need to do some installations of prerequisite software.

The best way to go about doing this is to register a task in the ServiceDefinition.csdef file. Lets imagine we need to run a batch file that will do some sort of installation, say a monitoring service that is required to be installed BEFORE IIS starts our web application, so that it can get a hook point, say New Relic!

Below is a configuration example that will do this for you.

https://gist.github.com/1775222

You can also set elevation privileges, which are required if you are running PowerShell scripts etc.

<Task commandLine="Startup.cmd" executionContext="elevated" taskType="background">

You can read more about this here:

http://msdn.microsoft.com/en-us/library/windowsazure/hh124132.aspx

http://msdn.microsoft.com/en-us/library/windowsazure/gg456327.aspx

http://msdn.microsoft.com/en-us/library/windowsazure/gg432991.aspx

So, I hope you now have a cool way to bootstrap your prerequisite software before IIS kicks in.

Windows Azure, which affinity group or region is best for you?

A nice tool to use to see which Azure Affinity Group to use e.g. South America, North America or Asia is to download this tool and run checks from where your clients will be based.

http://research.microsoft.com/en-us/downloads/5c8189b9-53aa-4d6a-a086-013d927e15a7/default.aspx

Once you got it installed, add your storage accounts and then get started.

image

So above we will test from Sydney Australia to our UAT environment in America.

Lets click β€œrun”

It will start executing the test, this is now a good time to plan a date, make a cup of coffee or write some JScript for your open source projects.

Results:

Sydney to North America

image

image

 

Sydney to South East Asia (Singapore)

image

image

Conclusion

For us, South East Asia was far more better (Web site, download is more important than upload), and the proof was in the pudding when we measures web site response times with MVC MiniProfiler.

However, this is not the ultimate conclusion, I bet these response times will vary depending on time of day, perhaps when Asia is awake and US is asleep, it could be the other way round, so test it at different times of day and pick the affinity or region that is best for you!