An Introduction to CloudFormation, AWS’s Infrastructure as Code

Being able to harness the power of a tool like CloudFormation can save you time, reduce errors and make managing your resources incredibly easy.

Infrastructure as code allows you to handle the set up of your infrastructure in the same way you would handle the development of your code: pick the right language or tool to do the job and start developing a solution that suits your needs, making it an executable specification that can be applied to target systems efficiently and repeatedly.

(DevOps for Developers, Michael Hüttermann)

When working with AWS, CloudFormation is potentially the language of choice for coding the infrastructure required for your platform or software.

The following example will create a VPC containing a public subnet and EC2 instance, a route table, IAM profile and role, security group, internet gateway and Elastic IP. You should be familiar with these services before continuing.

See the figure below for the intended outcome. This set up could be appropriate for a simple web hosting environment but could be easily expanded to contain private networks or other resources.

The process is relatively simple. Design and write a template for your infrastructure, and then use it to create a stack, which can then be maintained as required. A stack simply refers to the collection of created resources declared in the template.

Understanding the template

Templates are at the core working with CloudFormation. You have the option to write them in JSON or YAML formats. YAML is arguably easier to read and maintain when used for building templates, so will be used throughout this introduction. The template designer, discussed further on, does make it very easy to switch between the two formats at any point. YAML is essentially an indented JSON so the learning curve is very small if you are new to YAML.

The first level of the template is divided into several sections that serve different purposes. The only required section is Resources , the rest are optional. Parameters is potentially the most useful of the optional template sections, but this is dependent on the use case. Resources and Parameters will be the two sections discussed here.

You’ll also see a `MetaData` section, which is used by the designer to create the graphical representation of your resources. Click here for More info on template anatomy

Understanding how to use the template reference docs: resources page is invaluable to understanding this coding process. Find the resource you require within its service resource type section, i.e Instance within EC2. You now have a list of the properties available for that resource, how to define them, and if any are required. You can simply select the resources you need and build your template like lego.

Within the resource section, each resource has a key, which can be anything you want, followed by the parameters that define the resource.

Resources:
  MYVPCNAME:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

This structure repeats itself in all the other sections such as Parameters or Outputs, so you should become familiar with it.

As you start to add resources to the template, you will need to create references to define the links between resources. This is very easily achieved by completing the appropriate property with !Ref resourceName. The example below links the subnet to the VPC.

Resources:
  MYVPCNAME:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16  
  PublicSubnet:
    Type: 'AWS::EC2::Subnet'
    Properties:
      CidrBlock: 10.0.1.0/24
      VpcId: !Ref MYVPCNAME

This example has created a subnet within a VPC. This is a functioning template that you could now deploy as a stack. It’s not the most useful or exciting though, so let’s continue in more depth to harness the power of CloudFormation.

If you want to see the finished template used in the rest of this article click here.

Template designer

Understanding the designer and how to efficiently use it will really speed up your development process, and it is very easy to use.

Head to the AWS Console > CloudFormation > Create Stack. Under prepare template, select the radio button ‘create in designer’. You’ll be presented with a button below to launch the template designer.

The top half of the designer is your drawing board, with resources on the left column which can be dragged onto the board. The bottom half is the template itself. The tabs on this bottom section come in very useful. At the very bottom, you’ll see the quite self-explanatorily named ‘component’ and ‘template’ tabs.

The component tab allows you to easily edit the parameters of the selected resource, saving you from sifting through the entire template file, and possibly editing the wrong resource. Click on a resource in the drawing board, and it is automatically selected in the component tab.

It’s worth noting at this point that the designer does not automatically save. If you lose your connection or leave your desk for some time, you will likely lose your work. It’s worth getting in the habit of clicking the save icon frequently, or, my preferred method of copying and pasting the entire template into your text editor/IDE frequently. Using an IDE also give you the advantage of syntax highlighting, and managing your changes with a git repo.

Get Coding!

With an understanding of the template format, ability to interpret the reference docs to define resources, and an awareness of using the designer tool to generate the template, you should feel ready to get stuck into coding infrastructure that has some practical applications.

The following section will walk you through the steps needed to implement the following resources:

• VPC
• Public Subnet
• EC2 Instance
• EC2 Security Group
• IAM Instance Profile
• IAM Role
• Elastic IP
• Internet Gateway
• Route Table

Be sure to change the template language in the designer to YAML to understand the examples provided below.

The easiest way to begin building the template is to drag resources from the left-hand resource type column into the drawing board. All the resources used in this example, bar the IAM ones, can be found under the “EC2” resource type tab.

Unfortunately, there is no search feature to filter this huge list, but an easy way to figure out which resource you need is to have a tab with the EC2 template reference open. You’re likely going to need the reference open anyway to know what parameters to use, and, you can now use cmd+f/ctrl+f to search out the service you need.

Under EC2, drag VPC onto the drawing board. With it selected in the drawing board, click the component tab and change the name of the resource (the default will be some variation of EC2VPC#####). It’s good practice to change the key for each resource as it makes it far easier to reference resources, and generally read the template. Add the following to the properties of your VPC resource: CidrBlock: 10.0.0.0/16.

Every time you make a change, click the small tick near the top left to validate the template, then the refresh diagram button on the top right (not the browser refresh!). It’s good practice to do this frequently, along with saving or copying the template.

Drag a subnet from the EC2 resource types, and drop it inside the VPC. If you look in the template you’ll notice that the designer automatically completes a property VpcId with a value !Ref YOURVPCNAME for this subnet. This is an intrinsic function that provides the reference for resource’s properties that need to know about other resources.

If you drag the subnet outside the VPC in the drawing board, you’ll see an arrow appear representing this reference. The arrow is not necessary when the subnet is visually inside the VPC. Rename your subnet PublicSubnet and give it a CIDR block, CidrBlock: 10.0.1.0/24.

Your VPC and subnet should now be defined just like the example provided in the previous section about understanding the template.

Next, drag and drop an Instance into this subnet and rename it MainServer.

We’ll be adding some more values to the properties of this instance soon but for now, with the instance selected, in the component tab, replace the automatically generated NetworkInterfaces property and its values with the following:

Properties:
    ImageId: ami-05c424d59413a2876
    InstanceType: t2.micro
    SubnetId: !Ref PublicSubnet

We’ll be creating an elastic IP so don’t need to declare any NetworkInterfaces, but the SubnetId property launches the instance into the declared subnet. the ImageId determines what AMI the instances launches with. The above id is for Ubunutu. Choose the image and instance type that suits your needs.

The infrastructure so far:

Next up, create a security group to control access to the instance. Drag a security group from under the EC2 resource type, rename it ServerSecurityGroup.

In the drawing board, hover your mouse over the purple dots of the security group component. Click and drag the one that says VPC and release inside the VPC component. You’ll have noticed the VPC border went green, denoting that the selected property can be linked to it.

Click the security group component, and in the component tab, add the following to the security groups properties, to allow SSH and HTTP/S access:

Properties:
      VpcId: !Ref MYVPCNAME
      GroupDescription: Development server access over http/ssh
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          Description: SSH access
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          Description: HTTP access
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          Description: HTTPS access
          CidrIp: 0.0.0.0/0

You’ll notice CidrIp: 0.0.0.0/0 for the SSH which is not advisable. This will eventually be changed to reference a Parameter that will be created soon. Parameters allow user input/decisions to be made on stack creation giving some flexibility to the template, allowing you to set the permitted SSH IP when you’re ready. This also makes the template more reusable by not hardcoding a variable that is likely to change. Parameters will be discussed in more detail further on.

Remember to keep validating/refreshing/saving your template.

Hover over the purple dots of the Instance component, to find the SecuirtyGroup one, and drag it to the created security group. Your instance now has the security group applied to it.

Let’s provide the Instance with an ElasticIP for public access. Drop an EIP from the EC2 section inside the VPC, rename it ServerEIP . On the drawing board, drag the InstanceId property purple dot from the EIP to the Instance to assign it.

IAM Resources

The instance needs a profile to be able to access other AWS services. From the IAM resource types drop-down, drag an instance profile onto the board and rename it ServerProfile. Select the instance, and in the component tab add the property IamInstanceProfile: !Ref ServerProfile to apply the profile to the instance.

For a profile to work as intended in this use case, it needs a role, and the role needs policies attached to it. Drag a role from the IAM types section and rename it ServerRole. From the ServerProfile component on the drawing board, drag the role property to the created role.

Add the following properties to the role:

Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/AmazonEC2FullAccess'
        - 'arn:aws:iam::aws:policy/AmazonSESFullAccess'
        - 'arn:aws:iam::aws:policy/AmazonS3FullAccess'

The above is just an example of what policies you could attach to this role. Go to the AWS console > IAM > Policies to find the policies you want, and copy their respective arn’s to the ManagedPolicyArns property of your role.

Route Table and Gateway

These are the last resources needed for this infrastructure. From the EC2 resource types section, drag and drop a route table inside the VPC, renaming it RouteTable. Now, drag and drop a route inside the route table, select the route and in the component tab update it to the following:

PermitAllToIGW:
    Type: 'AWS::EC2::Route'
    Properties:
      RouteTableId: !Ref RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref IGW

This will route all traffic to the internet gateway. As your infrastructure grows, adding private subnets etc…, you’ll need to update this to reflect those modifications. For now, this will work suitably. From the subnet in the drawing board, select the SubnetRouteTableAssocciation property, and drag it to the PublicSubnet. The link you’ve just created actually exists as an independent resource, click on the black arrow connecting the table and subnet and rename it SubnetRouteTableAssociation.

Drag and drop an InternetGateway to above the VPC. Rename it IGW. Hover over the dots of the InternetGateway component in the drawing board to get the VPCGatewayAttachement property, and drag it into the VPC. This link also exists as a resource independent from the InternetGateway, click on the arrow and rename it IGWAttachement.

By this stage, you should have all the infrastructure’s resources in place, and it should look something like this:

Parameters

The final step of coding the template is adding some parameters. Parameters allow you to make decisions about any property you want at the stack creation phase. This really gives your template versatility and re-useability. These parameters could be IP addresses, AMI’s, or anything you need for your use case. Below DEVELOPERIP is used to define the SSH security group ingress rule so that the developer launching the stack will have SSH access to the instance. The second parameter will be SSHKEY, which will be applied to the instance to allow for an SSH connection. If you haven’t already got a key in your account, go to IAM >Users and upload an SSH key.

To implement these parameters, click on the template tab at the bottom of the screen. Scroll to the bottom of the template, and add the following to the very first level. Remember Parameters are a top-level section of the template.

Parameters:
  DEVELOPERIP:
    Description: 'Ip address for development machine to allow ssh, in CIDR format'
    Type: String
  SSHKEY:
    Description: Key pair for instance SSH access
    Type: 'AWS::EC2::KeyPair::KeyName'

You now need to create a reference in the resources to these parameters.

In the instance, under properties, add KeyName: !Ref SSHKEY.

In the security group under the ingress rule for port 22, update the CidrIp to CidrIp: !Ref DEVELOPERIP .

And that’s it! You should be ready to create a stack from this template and get straight into using your infrastructure. Save your template, then click on the cloud with the up arrow in it to create the stack. Your newly created template will be uploaded to an S3 bucket. Click next, choose a stack name, select your parameters and click next to configure options. Add a tag to make referencing your stack easy, and click next.

Review all, and click the confirm button about IAM resources. Create the stack!

You’ll now be able to watch all your hard work be rewarded as AWS builds your infrastructure. You may get some errors which will need debugging. Look through all the status reasons to figure out what happened if something went wrong.

Now you’ve got this intro under your belt, it’s time to start exploring and building more complex infrastructures, enjoy!

Here’s the completed template again as a reference.

Hope you’ve enjoyed this guide, I’d welcome any feedback on how to improve or to know how you got on with it. Just get in touch with me.

Bonus: UserData

As an extra, you can set the Instance property UserData in your template, or as a parameter to define a script that runs on the instance at launch. This section is potentially easier to write in JSON. Below is an example of how to define UserData within the template.

Some intrinsic functions are used to convert the text to Base64, and to join the lines declared in the array. Click here for more on intrinsic functions

In the designer, choose JSON as the template language. Click on the Instance in the drawing board, then the component tab in the bottom. From here you need to add UserData to the properties.

The structure to declare user data is :

UserData:
    {"Fn::Base64":
        {"Fn::Join":
            ["",
                [
                    "#/bin/bash \n",
                    "# one command per array item \n"
                ]
            ]
        }
    }

The following is an example of what it could look like:

This script will prepare the instance for use with AWS CodeDeploy and install NodeJS. Enjoy!