AWS CloudFormation is a template-based service to provision and manage infrastructure on AWS, consisting of a group of AWS resources such that the overhead tasks, such as describing the dependencies between AWS services or providing runtime parameters, are configurable in the template itself. The template is a reusable and declarative code approach to provisioning AWS infrastructure and is supported by a CloudFormation Designer, a GUI tool with provision to drag-and-drop components onto a diagram representing a template. In this article we shall discuss creating an RDS Oracle Database instance using CloudFormation. This article consists of the following sections:
- Overview of CloudFormation Template
- Setting the Environment
- Configuring a CloudFormation Template
- Creating a CloudFormation Stack
- Finding Status of RDS Instance
- Modifying and Adding Features to RDS Database Instance
- Storing CloudFormation Template in S3
Overview of CloudFormation Template
The CloudFormation template may be specified in JSON or YAML format and the following JSON document represents the structure of a template in which all the key/value pairs that may be specified are listed and the only required key is Resources. Each key value is a JSON object denoted with the {} notation.
{
"AWSTemplateFormatVersion" : "version date",
"Description" : "JSON string",
"Metadata" : {
},
"Parameters" : {
},
"Mappings" : {
},
"Conditions" : {
},
"Transform" : {
},
"Resources" : {
},
"Outputs" : {
}
}
The following listing represents CloudFormation template in YAML format.
---
AWSTemplateFormatVersion: "version date"
Description:
String
Metadata:
template metadata
Parameters:
Mappings:
Conditions:
Transform:
Resources:
Outputs:
Some of the template keys may be specified in any order even if values from keys defined later are used in keys defined earlier, but it is recommended to specify the keys in the order shown above, which follows the logical dependencies between sections. As an example, a Resource may make use of some parameters or mappings which are specified earlier in the template. Other keys are required to be specified in a particular order, as shown below in Table 1, which discusses each of the template keys.
Table 1. CloudFormation Template Keys
|
Key |
Description |
|
AWSTemplateFormatVersion |
Specifies the template version |
|
Description |
Template description as text string. Must be specified subsequent to AWSTemplateFormatVersion. |
|
Metadata |
Provides metadata information about the template. |
|
Parameters |
Specifies values that may be passed to template at runtime when a stack is created or updated. The parameters may be referred in Resources and Output keys. |
|
Mappings |
Mappings of keys/values to specify conditional parameters. The intrinsic function Fn::FindInMap may be used in Resources and Outputs section to look up key values specified in the Mappings. |
|
Conditions |
Specifies conditions for creating resources and assigning resource properties values. Used during stack creation and update. |
|
Transform |
Not relevant to RDS directly. Used for Lambda-based applications to specify the version of the AWS Serverless Application Model. |
|
Resources |
Specifies stack resources and their properties, which are specified with the Properties subsection for each resource. The resources may refer to other resources and resources may be referred in the Outputs. |
|
Outputs |
The stack’s properties that are output when describing stacks. |
Setting the Environment
The only prerequisite is an AWS account ,which may be created at https://aws.amazon.com/resources/create-account/. We need to provide a Virtual Private Cloud (VPC) for the CloudFormation, including Subnets in multiple zones for a Multi-Availability Zone CloudFormation. Create a VPC in AWS Console with Create VPC after selecting the VPC service from AWS services. The VPC ID listed in the Console shown in Figure 1 shall be used to the CloudFormation template.
Figure 1. AWS VPC
Create four Subnets associated with the VPC, one each in Availability Zone: us-east-1a,us-east-1b,us-east-1c, and us-east-1e, as shown in Figure 2.
Figure 2. Subnets in VPC
Configuring a CloudFormation Template
In this section we shall create a CloudFormation template (a .template file) to specify the resources, parameters, and other template keys to create an RDS database instance for Oracle Database. Set the "AWSTemplateFormatVersion" key to "2010-09-09" and add a Description, which could be a string to describe the template. In the Parameters section define the following (Table 2) parameters:
Table 2. Parameters
|
Parameter |
Description |
Default Value |
|
VpcId |
VPC ID of your existing Virtual Private Cloud (VPC) |
vpc-6a78780c |
|
Subnets |
The list of Subnet Ids, for at least two Availability Zones in the region in your Virtual Private Cloud (VPC) |
subnet-10461675,subnet-dff72197,subnet-b50ce899,subnet-2b077c17 |
|
DBName |
The Oracle database name |
orcldb |
|
DBUsername |
The Oracle database admin account username |
orcl |
|
DBPassword |
The Oracle database admin account password |
orcl12crds |
Next, define mappings in Mappings section for AWS instance type to architecture AWSInstanceType2Arch. For t2.micro instance type, set mapping as follows.
"t2.micro": {
"Arch": "64"
}
In the Resource section define resources listed in Table 3.
Table 3. Resources
|
Resource |
Type |
Description |
|
OrclDBSubnetGroup |
AWS::RDS::DBSubnetGroup |
Subnets available for the RDS DB Instance |
|
OrclVPCSecurityGroup |
AWS::EC2::SecurityGroup |
Security group for RDS DB Instance |
|
OraDB |
AWS::RDS::DBInstance |
RDS Database instance |
|
OrclRDSParamGroup |
AWS::RDS::DBParameterGroup |
Oracle DB Parameter Group |
The properties for each of the resources are specified with the Properties key. The properties for the OrclDBSubnetGroup resource in addition to the DBSubnetGroupDescription property are listed in Table 4.
Table 4. OrclDBSubnetGroup resource Properties
|
Property |
Description |
Value |
|
SubnetIds |
Subnet Ids |
{"Ref": "Subnets"}
|
The properties for the OrclVPCSecurityGroup resource, in addition to GroupDescription, are discussed in Table 5.
Table 5. OrclVPCSecurityGroup resource Properties
|
Property |
Description |
Value |
|
VpcId |
VPC Id |
{"Ref": "VpcId"} |
|
SecurityGroupIngress |
Security group Inbound rules as an array. An inbound rule to allow traffic of any IP protocol from any port and to any port must be included with the “-1” value. |
[{ "IpProtocol": "tcp", "FromPort": "80", "ToPort": "80", "CidrIp": "0.0.0.0/0" }, { "IpProtocol": "tcp", "FromPort": "22", "ToPort": "22", "CidrIp": "0.0.0.0/0" },{ "IpProtocol": "-1", "FromPort": "-1", "ToPort": "-1", "CidrIp": "0.0.0.0/0" }
]
|
The OraDB resource has the properties discussed in Table 6.
Table 6. OraDB resource Properties
|
Property |
Description |
Value |
|
DBName |
Oracle database name |
{"Ref": "DBName"} |
|
DBInstanceIdentifier |
Database instance identifier |
"orcl" |
|
DBSubnetGroupName |
Database Subnet group |
{"Ref": "OrclDBSubnetGroup"} |
|
VPCSecurityGroups |
VPC Security groups |
[{"Ref": "OrclVPCSecurityGroup"}] |
|
AllocatedStorage |
Allocate storage for RDS DB instance |
“100” |
|
DBInstanceClass |
DB Instance Class |
“db.t2.micro” |
|
Engine |
Database Engine |
"oracle-ee" |
|
EngineVersion |
Database Engine version |
"12.1.0.2.v7" |
|
LicenseModel |
License model |
"bring-your-own-license" |
|
Port |
Database listener port |
1521 |
|
MasterUsername |
Database Master user name |
{"Ref": "DBUsername"} |
|
MasterUserPassword |
Database Master Password |
{"Ref": "DBPassword"} |
|
DBParameterGroupName |
Database Parameter Group |
{"Ref": "OrclRDSParamGroup"} |
|
MultiAZ |
Whether the RDS resource is multi availability zone (AZ). In a multi AZ deployment a database update incurs no interruption. |
“true” |
Additionally, the DeletionPolicy for the AWS::RDS::DBInstance type resource is set to Snapshot, which implies that a snapshot is created when the resource is deleted.
The OrclRDSParamGroup resource has the properties discussed in Table 7 along with the Description property.
Table 7. OrclRDSParamGroup resource Properties
|
Property |
Description |
Value |
|
Family |
Parameter group family |
oracle-ee-12.1 |
|
Parameters |
Parameters includes the compatibility version for Oracle database |
{"compatible": "12.1.0.2"} |
The Outputs section defines a connection string for JDBC connection to the RDS database instance. The template for creating a stack for a RDS database instance for Oracle Database is listed:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation Template Oracle RDS",
"Parameters": {
"VpcId": {
"Default": "vpc-6a78780c",
"Type": "String",
"Description": "VpcId of your existing Virtual Private Cloud (VPC)"
},
"Subnets": {
"Default": "subnet-10461675,subnet-dff72197,subnet-b50ce899,subnet-2b077c17",
"Type": "CommaDelimitedList",
"Description": "The list of SubnetIds, for at least two Availability Zones in the region in your Virtual Private Cloud (VPC)"
},
"DBName": {
"Default": "orcldb",
"Description": "The Oracle database name",
"Type": "String",
"MinLength": "1",
"MaxLength": "64",
"AllowedPattern": "[a-zA-Z][a-zA-Z0-9]*",
"ConstraintDescription": "must begin with a letter and contain only alphanumeric characters."
},
"DBUsername": {
"Default": "orcl",
"NoEcho": "true",
"Description": "The Oracle database admin account username",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern": "[a-zA-Z][a-zA-Z0-9]*",
"ConstraintDescription": "must begin with a letter and contain only alphanumeric characters."
},
"DBPassword": {
"Default": "orcl12crds",
"NoEcho": "true",
"Description": "The Oracle database admin account password",
"Type": "String",
"MinLength": "8",
"MaxLength": "41",
"AllowedPattern": "[a-zA-Z0-9]*",
"ConstraintDescription": "must contain only alphanumeric characters."
}
},
"Mappings": {
"AWSInstanceType2Arch": {
"t1.micro": {
"Arch": "64"
},
"t2.micro": {
"Arch": "64"
},
"m1.small": {
"Arch": "64"
}
}
},
"Resources": {
"OrclDBSubnetGroup": {
"Type": "AWS::RDS::DBSubnetGroup",
"Properties": {
"DBSubnetGroupDescription": "Subnets available for the RDS DB Instance",
"SubnetIds": {
"Ref": "Subnets"
}
}
},
"OrclVPCSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Security group for RDS DB Instance.",
"VpcId": {
"Ref": "VpcId"
},
"SecurityGroupIngress": [{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": "0.0.0.0/0"
},{
"IpProtocol": "-1",
"FromPort": "-1",
"ToPort": "-1",
"CidrIp": "0.0.0.0/0"
}
]
}
},
"OraDB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName": {
"Ref": "DBName"
},
"DBInstanceIdentifier": "orcl",
"DBSubnetGroupName": {
"Ref": "OrclDBSubnetGroup"
},
"VPCSecurityGroups": [{
"Ref": "OrclVPCSecurityGroup"
}],
"AllocatedStorage": "100",
"DBInstanceClass": "db.t2.micro",
"Engine": "oracle-ee",
"EngineVersion": "12.1.0.2.v7",
"LicenseModel": "bring-your-own-license",
"Port": "1521",
"MasterUsername": {
"Ref": "DBUsername"
},
"MasterUserPassword": {
"Ref": "DBPassword"
},
"DBParameterGroupName": {
"Ref": "OrclRDSParamGroup"
},
"MultiAZ": "true"
},
"DeletionPolicy": "Snapshot"
},
"OrclRDSParamGroup": {
"Type": "AWS::RDS::DBParameterGroup",
"Properties": {
"Family": "oracle-ee-12.1",
"Description": "Oracle DB Parameter Group",
"Parameters": {
"compatible": "12.1.0.2"
}
}
}
},
"Outputs": {
"JDBCConnectionString": {
"Description": "JDBC connection string for the Oracle database",
"Value": {
"Fn::Join": [
"", [
"jdbc:oracle://",
{
"Fn::GetAtt": [
"OraDB",
"Endpoint.Address"
]
},
":",
{
"Fn::GetAtt": [
"OraDB",
"Endpoint.Port"
]
},
"/",
{
"Ref": "DBName"
}
]
]
}
}
}
}
Creating a CloudFormation Stack
In this section we shall create a CloudFormation Stack for an RDS database instance. Select the AWS CloudFormation service in the AWS Console or with URL https://console.aws.amazon.com/cloudformation. Click on Create New Stack as shown in Figure 3.
Figure 3. Create New Stack for CloudFormation
In Select Template click on Design a template to design a template as shown in Figure 4. Alternatively, a sample template may be selected or a template may be uploaded to S3, or an S3 template URL may be specified.
Figure 4. Design Template
The AWS CloudFormation Designer gets started, as shown in Figure 5.
Figure 5. AWS CloudFormation Designer
Several options are available for designing a template:
- A template may be created using the Designer by selecting components from the resource types palette and dragging them on to the Designer Components layout section/region. Resources may be linked in the Designer.
- The template may be copied and pasted into the Designer by selecting the Template tab
- A template may be opened from an AWS S3 Bucket
We shall open a template that is stored in S3 storage. Select the document/file icon and click on Open, as shown in Figure 6.
Figure 6. Selecting Open to open a CloudFormation Template
In the Open a template dialog click on Amazon S3 bucket as shown in Figure 7.
Figure 7. Open a template>Amazon S3 bucket
Two options get listed to select a template, as shown in Figure 8.
- Template URL
- Select a template file
Click the option Select a template file.
Figure 8. Options for selecting a CloudFormation Template from S3 Bucket
Select the template, and click on Open as shown in Figure 9.
Figure 9. Opening a Template
The template gets opened and a diagram gets displayed in the CloudFormation Designer, as shown in Figure 10. The Template tab is selected by default, the other tab being Components.
Figure 10. Template in CloudFormation Designer
Specify a file name (oracle-rds.template) for the template and click on Save as shown in Figure 11.
Figure 11. Updating a template file name and saving a Template File
Select the Components tab to display the components. Various tabs are provided for the template components: Parameters, Mappings, Conditions, Metadata, and Outputs. Click on the icon for Validate template to validate the template as shown in Figure 12.
Figure 12. Validating the Template
The template is only a starting blueprint for a CloudFormation stack. Click on Create stack to create a new Stack, as shown in Figure 13.
Figure 13. Create stack
The Create stack page from which the Design template option was selected gets re-displayed with the Amazon S3 template URL added to the field, which was previously empty. Click on Next as shown in Figure 14.
Figure 14. Selecting a Template
In the Specify Details page specify a Stack name (orcl-rds). Most of the other fields are picked from the template if specified in the template, as shown in Figure 15. Click on Next.
Figure 15. Specifying Stack Details
Specify any Options and click on next as shown in Figure 16. We have not specified any options.
Figure 16. Specifying Options
In the Review page, the Summary of the Stack is displayed. Click on Create as shown in Figure 17.
Figure 17. Review>Create
In the page from which Create New Stack was selected, click on the Refresh icon, as shown in Figure 18 to refresh the Stacks.
Figure 18. Refreshing Stacks
The orcl-db Stack is listed with the Status Create_In_Progress, as shown in Figure 19. Click on the stack link.
Figure 19. New Stack
Stack detail gets listed, including Stack name, ID and status as shown in Figure 20. Click on Stacks in the toolbar.
Figure 20. Stack Detail
Tabs for Overview, Outputs, Resources, Events, Template, Parameters, Tags, Stack Policy, and Change Sets get displayed, as shown in Figure 21.The stack Overview is selected by default.
Figure 21. Stack Overview and Tabs for Stack sections
Select the Resources tab for the stack resources. Some of the resources are listed as Completed while others are in the process of being created, as shown in Figure 22.
Figure 22. Stack Resources
Click on the Events tab to list the events, as shown in Figure 23. Event information includes a timestamp, Status, Type, and Logical ID.
Figure 23. Stack Events
Click on the Parameters tab to list the Stack parameters, as shown in Figure 24.
Figure 24. Stack Parameters
The template may be listed with the Template tab, as shown in Figure 25.
Figure 25. Template Tab
When all the Stack resources have been created the stack Status becomes CREATE_COMPLETE, as shown in Figure 26.
Figure 26. Stack created and all Resources with Status CREATE_COMPLETE
When the stack has been completed the Events should list the Status CREATE_COMPLETE for the AWS::CloudFormation::Stack type event as shown in Figure 27.
Figure 27. All Event Types CREATE_COMPLETE
The Overview also lists the Stack Status as CREATE_COMPLETE, as shown in Figure 28.
Figure 28. Overview Status as CREATE_COMPLETE
Finding Status of RDS Instance
After the Stack lists the status as CREATE_COMPLETE select the RDS service in the AWS Console and click on the refresh icon in the RDS Console to list the RDS DB instances. The orcl DB Instance is listed with Status “creating” as shown in Figure 29.
Figure 29. RDS DB Instance orcl created with CloudFormation
The RDS DB instance creation could take a varying amount of time depending on the RDS service create latency. At times an increased instance create latency may cause up to an hour or more for the RDS DB instance to get created and become available. Select the Notifications to find if any "Increased Instance Create Latencies" exist or have been resolved, as shown in Figure 30.
Figure 30. Notifications about Increased Instance Create Latencies
When the RDS instance becomes created the Status becomes “available” and the Endpoint becomes listed, as shown in Figure 31.
Figure 31. RDS DB Instance Available
The Configuration Details include the DB Name, Availability Zone, VPC, Subnets and the Endpoint, as shown in Figure 32.
Figure 32. Configuration Details
Modifying and Adding Features to RDS Database Instance
An RDS instance that does not get created due the omission of some configuration information in the CloudFormation template may still be created by providing the missing configuration information in the RDS Console itself. As an example, omit the following SecutiryGroupIngress, which is required for Inbound permissions to the RDS instance, from the template from which the stack is created. As the stack was already successfully completed, a new stack would need to be created with the SecurityGroup Ingress omitted.
{
"IpProtocol": "-1",
"FromPort": "-1",
"ToPort": "-1",
"CidrIp": "0.0.0.0/0"
}
The RDS instance launched from the template would not get created and available and instead the “No Inbound Permissions” error gets displayed in the RDS Console as shown in Figure 33.
Figure 33. An RDS Instance with No Inbound Permissions
Click on the triangular icon to display detail about the error condition in a dialog as shown in Figure 34. As the dialog indicates, Security Group Configuration Does not Allow Inbound Connections.
Figure 34. Displaying Error Dialog for No Inbound Permissions
The Security Group may be edited and the required Inbound Permissions added to make the RDS instance available. Click on Edit Security Group as shown in Figure 35.
Figure 35. Edit Security Group
The Security Group for the RDS instance gets opened in the RDS Console as shown in Figure 36. Click on the Edit button to add the Inbound Permissions.
Figure 36. Security Group>Edit
Click on Add Rule to add a new rule or modify the rule listed to make Type as All traffic, Protocol as All, Port Range as 0-65535, Source as Anywhere, as shown in Figure 37. Click on Save.
Figure 37. Adding or Modifying Inbound Permissions
The Inbound Rules should include all traffic for all protocols, for all port ranges and all sources as shown in Figure 38.
Figure 38. Modified Inbound Rules
Click on the Refresh Data icon as shown in Figure 39.
Figure 39. Refreshing Data
The Connection Information gets updated, as shown in Figure 40.
Figure 40. Connection Information Updated
The Endpoint which previously indicated No Inbound Permissions becomes authorized as shown in Figure 41. The RDS instance becomes available.
Figure 41. Endpoint and RDS Instance Available
The Stack status which was CREATE_IN_PROGRESS due to the RDS instance not being created due to No Inbound Permissions” becomes CREATE_COMPLETE, as shown in Figure 42.
Figure 42. Stack Status becomes CREATE_COMPLETE
Storing CloudFormation Template in S3
The template for a Stack that is CREATE_COMPLETE may be saved in an S3 Bucket to be reused. Click on Actions>View/Edit template in a Designer as shown in Figure 43.
Figure 43. Actions>View/Edit Template in Designer
The template gets opened in CloudFormation Designer. Click on the document icon and select Save as shown in Figure 44.
Figure 44. Saving a Template
In the Save your template dialog click on Amazon S3 bucket as shown in Figure 45.
Figure 45. Save your template>Amazon S3 bucket
Specify a File name (orcl-rds) and click on Save as shown in Figure 46.
Figure 46. Saving template in Amazon S3 bucket
If the template gets saved the message “Your template has been saved in the following location” gets displayed, as shown in Figure 47.
Figure 47. Template Saved in S3 Bucket
In this article we discussed a declarative coding approach to provision AWS Infrastructure for an RDS DB instance. The Infrastructure as Declarative Code approach is suitable if fine-grained configuration of Infrastructure for a database is to be made. AWS CloudFormation may be used not only for an RDS DB instance but also other databases
Start the discussion at forums.toadworld.com