Create a hardened Ubuntu Pro 18.04 LTS shared image with Azure Image Builder

1. Overview

In this tutorial, we will use Azure Image Builder to create a hardened Ubuntu Pro 18.04 LTS “golden” image in an Azure Shared Image Gallery.

The resulting images will have CIS hardening applied to them, which helps meet security best practice, CIS-specific requirements and also improves compliance with the Azure Linux Security Baseline policy.

What you’ll learn

  • How to set up your Azure environment with a Shared Image Gallery and Azure resources you need to distribute the image within your environment
  • How to create an image definition for Ubuntu Pro 18.04 LTS and customise the image build JSON to apply the CIS hardening and add any other applications we may want in every Ubuntu VM
  • How to create the image version in this Azure Image Builder service
  • How to create a VM from the image in the Shared Image Gallery

We will be using Ubuntu Pro as the starting point for our Ubuntu images. These are normally the best choice for production workloads on Azure and they include access to CIS hardening scripts.

What you’ll need

Credits

This tutorial is based on the article Preview: Create a Linux image and distribute it to a Shared Image Gallery by using Azure CLI in the Microsoft documentation and an earlier version of this tutorial created by David Coronel here.


3. Customise a template for our deployment

Now we are going to create a template that contains the build instructions for the standard, “golden” image we want to create.

We can download a starting point for this from here

wget https://gist.githubusercontent.com/Hooloovoo/3e544681a12121a36b5dbda684465b8d/raw/85a7c4461e704b722b901ffc981e23db9a14ce82/BasicCISUbuntuPro1804SIGTemplate.json

Then we can customise it to use the values we have set above. The sed commands below simply replace the <variable> placeholders in BasicCISUbuntuPro1804SIGTemplate.json with the values for the parameters that we set earlier:

sed -i -e "s/<subscriptionID>/$subscriptionID/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<rgName>/$sigResourceGroup/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<imageDefName>/$imageDefName/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<sharedImageGalName>/$sigName/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<region1>/$location/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<region2>/$additionalregion/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<runOutputName>/$runOutputName/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s%<imgBuilderId>%$imgBuilderId%g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<ProPlanPublisher>/$ProPlanPublisher/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<ProPlanOffer>/$ProPlanOffer/g" BasicCISUbuntuPro1804SIGTemplate.json
sed -i -e "s/<ProPlanSku>/$ProPlanSku/g" BasicCISUbuntuPro1804SIGTemplate.json

4. Review the contents of the template file

Let’s review some of the sections of the BasicCISUbuntuPro1804SIGTemplate.json we have just updated. It is worth reading over all of this (short) file, but below we will look at particular sections in more detail. We should not need to change any of these values.

Note the below section:

        "source": {
            "type": "PlatformImage",
                "publisher": "canonical",
                "offer": "0001-com-ubuntu-pro-bionic",
                "sku": "pro-18_04-lts",
                "version": "latest",
        "planInfo": {
                    "planName": "pro-18_04-lts",
                    "planProduct": "0001-com-ubuntu-pro-bionic",
                    "planPublisher": "canonical"
                }
        },

This will show the plan details for the Marketplace or Private Offer VM image you are using as a starting point for your golden image.

The customize section allows us to run commands as part of the image building process. The following waits until Ubuntu’s ua client has attached to its subscription and then enables access to the CIS hardening scripts:

        "customize": [
            {
            "type": "Shell",
            "name": "WaitForUAtokenAutoAttach",
            "inline": [
                "sudo ua status --wait"
            ]
        },

        {
            "type": "Shell",
            "name": "EnableCISfeature",
            "inline": [
            	"sudo ua enable cis"
            ]
        },

In a real deployment, we would at this point in the customisation script modify /usr/share/ubuntu-scap-security-guides/cis-hardening/ruleset-params.conf to set values appropriate for our environment, before running the CIS hardening script. See this documentation for more information. See also the rest of the CIS documentation here. To keep things simple, however, we will skip that step here.

The following runs the CIS hardening script and then removes one of these rules, as it conflicts with how Azure provides provisioning information to VMs:

        {
            "type": "Shell",
            "name": "RunCIShardening - see https://ubuntu.com/security/certifications/docs/cis-compliance",
            "inline": [
                "sudo /usr/share/ubuntu-scap-security-guides/cis-hardening/Canonical_Ubuntu_18.04_CIS-harden.sh lvl1_server"
            ]
        },

        {
            "type": "Shell",
            "name": "UDFworkaroundForAzureVMbooting - UDF is required for Azure image provisioning",
            "inline": [
                "sudo rm -f /etc/modprobe.d/Canonical_Ubuntu_CIS_rule-1.1.1.7.conf"
            ]
        },

The following is a placeholder for any custom commands we want to run. We will come back to this in the next step.

        {
            "type": "Shell",
            "name": "Placeholder for custom commands required in each Ubuntu VM",
            "inline": [
                "echo 'Replace me!'"
            ]
        },

The below runs some commands that deregisters the golden image from Ubuntu Pro and removes the machine-id. This ensures that VMs generated from the golden image will generate their own unique IDs.

	    {
            "type": "Shell",
            "name": "DetachUA -- images created from this will auto attach themselves with new credentials",
            "inline": [
                "sudo ua detach --assume-yes && sudo rm -rf /var/log/ubuntu-advantage.log"
            ]
     	},

        {
            "type": "Shell",
            "name": "Replace /etc/machine-id with empty file to ensure UA client does not see clones as duplicates",
            "inline": [
                "sudo rm -f /etc/machine-id && sudo touch /etc/machine-id"
            ]
        }

Custom customisation: Make the VMs auto-update

As an example of how we can customise an image building script for our requirements, we will now change one of these customize steps.

Find the following section in BasicCISUbuntuPro1804SIGTemplate.json:

        {
            "type": "Shell",
            "name": "Placeholder for custom commands required in each Ubuntu VM",
            "inline": [
                "echo 'Replace me!'"
            ]
        },

Change this to the below, which enables unattended upgrades on the machine:

        {
            "type": "Shell",
            "name": "Install upgrades automatically",
            "inline": [
                "sudo apt install unattended-upgrades"
            ]
        },

5. Build/Create the image version

We will now create the image version in the gallery.

First, we submit the image configuration to the Azure Image Builder service:

az resource create \
    --resource-group $sigResourceGroup \
    --subscription $subscriptionID \
    --properties @BasicCISUbuntuPro1804SIGTemplate.json \
    --is-full-object \
    --resource-type Microsoft.VirtualMachineImages/imageTemplates \
    -n BasicCISUbuntuPro1804SIG01

In the next step we will start the image build. This step can take many minutes (25-30 mins on my testing), as Azure will actually launch a VM and run the steps we have defined. We need to wait for this to complete before we can create a VM.

az resource invoke-action \
     --resource-group $sigResourceGroup \
     --subscription $subscriptionID \
     --resource-type  Microsoft.VirtualMachineImages/imageTemplates \
     -n BasicCISUbuntuPro1804SIG01 \
     --action Run

If this fails, see the section below
If this fails, see the “If required: Accept Marketplace terms and retry” section below

While we are waiting, however, we can see the logs of the AIB build process by going to the storage account inside the resource group created by AIB (ie. Azure Portal > Resource groups > [IT_ibUbuntuProGalleryRG_BasicCISUbuntuPro18_randomID > Random ID of the storage account > Containers > packerlogs > Random ID of the container > customization.log > Download.

You should be able to see traces of the CIS hardening like this:

[...]
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Execute rule 1.1.1.2
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm:
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Ensure mounting of freevxfs filesytems is disabled
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Execute rule 1.1.1.3
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm:
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Ensure mounting of jffs2 filesytems is disabled
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Execute rule 1.1.1.4
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm:
[4c6d22ac-7c0f-466f-a1c2-b30672274e89] PACKER OUT     azure-arm: Ensure mounting of hfs filesytems is disabled
[...]

If required: Accept Marketplace terms and retry

If you have never launched an Ubuntu Pro 18.04 LTS image before, this command may fail with something like:

Deployment failed. Correlation ID: eba17586-86f5-495a-85f9-3f9191065447. During the image build a failure has occurred. Please review the build log to identify which build/customization step failed. For more troubleshooting steps go to https://aka.ms/azvmimagebuilderts. Image build log location: https://armecifcu32rwz7jlos8zurq.blob.core.windows.net/packerlogs/bf091959-ba35-4db2-8888-c2b3179f119e/customization.log. OperationId: 1e853ba4-7d46-493f-890e-7e039fa44f9f. Use this operationId to search packer logs.

and if you look at the logs it mentions, you can see something like:

{\"code\":\"MarketplacePurchaseEligibilityFailed\",\"message\":\"Marketplace purchase eligibilty check returned errors. See inner errors for details. \",\"details\":[{\"code\":\"BadRequest\",\"message\":\"Offer with PublisherId: 'canonical', OfferId: '0001-com-ubuntu-pro-bionic' cannot be purchased due to validation errors. For more information see details. [...] 
You have not accepted the legal terms on this subscription: '[subscription ID]' for this plan. Before the subscription can be used, you need to accept the legal terms of the image. To read and accept legal terms, use the Azure CLI commands described at https://go.microsoft.com/fwlink/?[REDACTED] or the PowerShell commands available at https://go.microsoft.com/fwlink/?[REDACTED] Alternatively, deploying via the Azure portal provides a UI experience for reading and accepting the legal terms. Offer details: publisher='canonical' offer = '0001-com-ubuntu-pro-bionic', sku = 'pro-18_04-lts', Correlation Id: '7c302144-1c69-4642-be15-10e75049d0f7'.

In this case, we will need to accept the terms and retry the above command. You can see the terms on the Ubuntu Pro 18.04 Pro LTS marketplace listing here or for your specific Private Offer.

To accept, you can either launch a single VM through the Portal for a UI experience or you can accept from the commandline with:

az vm image terms accept --plan $ProPlanSku --offer $ProPlanOffer --publisher $ProPlanPublisher --subscription $subscriptionID

Then we need to re-run:

az resource invoke-action \
     --resource-group $sigResourceGroup \
     --subscription $subscriptionID \
     --resource-type  Microsoft.VirtualMachineImages/imageTemplates \
     -n BasicCISUbuntuPro1804SIG01 \
     --action Run

(If it does not work immediately, wait a few minutes and try again.)

Completing the build process

Once it has completed, it will change from “Running” to show something like the following:

{
  "endTime": "2021-11-03T17:42:21.7582048Z",
  "name": "67B28A17-AD58-4BDD-8CFB-1777906F8626",
  "startTime": "2021-11-03T17:11:38.9933333Z",
  "status": "Succeeded"
}

8. Cleanup

You should be able to see the Resource Groups that have been created as part of this tutorial by typing:

az group list --query [].name --output table --subscription $subscriptionID | grep $sigResourceGroup

In my case this returns:

ibUbuntuProGalleryRG
IT_ibUbuntuProGalleryRG_BasicCISUbuntuPro18_e52db49b-9cc0-4d52-936e-ecf570746def

Check you are happy for these to be deleted. If so, type:

az group delete --name [the name from above] --subscription $subscriptionID

for each of these Resource Groups, for example:

$ az group delete --name ibUbuntuProGalleryRG --subscription $subscriptionID
Are you sure you want to perform this operation? (y/n): y
$ az group delete --name IT_ibUbuntuProGalleryRG_BasicCISUbuntuPro18_e52db49b-9cc0-4d52-936e-ecf570746def --subscription $subscriptionID
Are you sure you want to perform this operation? (y/n): y

(You may find that deleting the first automatically deletes the second.)


9. That’s all folks!

Congratulations! We have created a Shared Image Gallery with a hardened Ubuntu Pro 18.04 LTS image inside, and launched and tested virtual machines created from this.

I hope that you have found this a helpful introduction to using a Shared Image Gallery with your Ubuntu Pro entitlements. If you have any questions, comments or suggestions, please do click the “Suggest changes” link and comment on the Discourse article for this tutorial.