Monitoring and Evaluating Database Configurations with AWS Config – Part I

Introduction

Consider the use case that multiple AWS resources are running to support applications running on AWS. (We shall mainly be concerned with the AWS databases RDS, Redshift, and DynamoDB.) An administrator would be interested in finding out if the applications are running at the desired configurations.

Best Practice for Evaluating Database Configurations

While it is possible to navigate to the AWS Management Console for each database and find out about configuration settings, a tool that monitors and records configurations on all AWS resources for continuous evaluation would be most desirable. AWS Config is such a tool. Using AWS Config, an administrator is able to perform the following configurations related tasks.

• Set desired configurations for AWS resources and let AWS Config automatically discover and evaluate the actual recorded configurations in comparison to the desired configurations. AWS Config has the provision to codify compliance with custom Config rules in AWS Lambda.
• Review the changes or updates in configurations
• Find details about resource configurations recorded in the past, including for resources that have been deleted and resources being recorded in real-time
• Dynamic Compliance auditing, which involves determining compliance of the actual configurations with the desired configurations
• Continuous change management to determine the resource dependencies of different resources, including how modifying one resource configuration affects another resource
• Integration with other AWS services. Configuration changes are streamed to Simple Notification Services (SNS). The compliance may be codified with custom settings using AWS Lambda. Configuration changes are saved in AWS S3.

The typical methodology used by AWS Config is as follows.

1. AWS Administrator sets the desired configurations for the AWS resources being used.
2. Configuration changes are made or occur automatically in AWS resources.
3. AWS Config discovers the configuration changes and records them in a normalized format.
4. Normalized changes are stored in S3, sent to Topic/s and Subscriptions configured in Simple Notification Service (SNS).
5. Config evaluates the actual recorded configurations with the desired configurations.
6. Evaluations are made available on the Config Dashboard and may also be sent to SNS topics and subscriptions. Config APIs are also available to access the normalized changes and evaluations made by Config.

AWS Config provides managed rules that are used to evaluate AWS resources. AWS Config managed rules are provides in several categories: Compute (AMIs, EC2, and EBS), Database (DynamoDB, RDS, and Redshift), Management Tools (CloudTrail, CloudWatch), Security (IAM), and Storage (S3). Custom Config rules, which may extend managed rules already available, may also be defined.

In this article we shall discuss using AWS Config to monitor configurations of databases on AWS and make evaluations and compliance auditing of the configurations. This two-article series has the following sections.

• Setting the Environment
• Configuring AWS Config Global Settings
• Creating RDS, Redshift and DynamoDB Instances
• Adding AWS Config Rules
• Adding and Evaluating AWS Config Rules for DynamoDB
• Adding and Evaluating AWS Config Rules for Databases on RDS
• Adding and Evaluating AWS Config Rules for Redshift
• Finding and Listing Resources that are managed by AWS Config

Setting the Environment

The only prerequisite is an AWS account, which may be created at https://aws.amazon.com/resources/create-account/ .

Configuring AWS Config Global Settings

Open the AWS Config Console at https://aws.amazon.com/config/ and select Settings in the margin as shown in Figure 1. Recording is on by default. The Turn off button is provided to turn off the recording. By default Resource types to record is set to All resources, as shown in Figure 1.

Figure 1. AWS Config Settings

Select or create an Amazon S3 bucket to store configuration history and snapshot files with details for the resources that AWS Config records. Choose the config-bucket- prefixed bucket as shown in Figure 2.

Figure 2. Choosing a S3 Bucket

In Amazon SNS Topic choose the option Stream configuration changes and notifications to an Amazon SNS topic as shown in Figure 3. Select the option Choose a topic from your account and select the Topic nameconfig-topic.

Figure 3. Configuring a SNS Topic for AWS Config

Select the option Choose a role from your account to grant AWS Config read-only access to AWS resources so that it may record configuration changes, and also grant Config permissions to send the configuration information to S3, as shown in Figure 4. Select the Role name config-role-us-east-1; the role name could be different for different users. Click on Save.

Figure 4. Selecting an AWS Config Role Name

The AWS Config Settings get saved, as shown in Figure 5 by the Success message.

Figure 5. AWS Config Settings Saved

Creating RDS, Redshift and DynamoDB Instances

Next, create AWS database resources on which to record the configuration changes. Create a few RDS Database instances, a DynamoDB instance and a Redshift instance. Within RDS some of the configurations are different for the different RDS engine types. Select from the different type of RDS Engines as shown in Figure 6.

Figure 6. Selecting a RDS Engine

Also, configurations are different for the different editions of an RDS Engine. As an example, the configurations are different for the different editions of Oracle Database: Oracle EE, Oracle SE, and Oracle SE One, as shown in Figure 7.

Figure 7. Selecting a Oracle Database Edition

The configurations are also different based on whether Production or Dev/Test is selected as the deployment mode, as shown in Figure 8.

Figure 8. Selection a Production or Dev/Test Deployment Mode

RDS Console with an Aurora DB cluster and replica, two Oracle DB instances, and a MySQL DB instance is shown in Figure 9. Different users may have different RDS databases.

Figure 9. RDS Console with DB Instances

Create a DynamoDB instance in the DynamoDB Console as shown in Figure 10.

Figure 10. DynamoDB Instance

Create a Redshift Cluster as shown in Figure 11. The procedure to create a Redshift cluster is discussed in an earlier article on using Redshift.

Figure 11. Redshift Cluster

AWS Config is used for evaluating compliance of AWS services with the configured settings. In this article we introduced AWS Config and set the environment by creating database instances for RDS, Redshift and DynamoDB. In a subsequent article we shall add AWS Config rules to evaluate databases.