Manage AWS RDS instances

14min
|
Terraform

AWS's Relational Database Service (RDS) provides hosted relational databases, which are easier to operate and maintain than self-managed implementations. Terraform can provision, scale, and modify RDS, enabling you to manage the RDS instance and cluster life cycle programmatically, safely, and declaratively.

In this tutorial, you will use Terraform to provision an RDS instance, subnet group, and parameter group, modify the RDS instance configuration, and provision a replica instance.

Prerequisites

This tutorial assumes that you are familiar with the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first.

For this tutorial, you will need:

Terraform 0.14+ installed locally
an AWS account with credentials configured for Terraform
psql 14.2+

Note

Some of the infrastructure in this tutorial may not qualify for the AWS free tier. Destroy the infrastructure at the end of the guide to avoid unnecessary charges. We are not responsible for any charges that you incur.

Clone the sample repository

Clone the sample repository for this tutorial, which contains Terraform configuration for an RDS instance, parameter group, security group, and subnet group.

$ git clone https://github.com/hashicorp-education/learn-terraform-rds

Change into the repository directory.

$ cd learn-terraform-rds

Review resource configuration

Open the main.tf file in your editor to review the sample configuration.

Networking components

The first resources defined are the VPC and subnets, using the terraform-aws-vpc module.

Warning

For simplicity, this RDS tutorial instance is publicly accessible. Avoid configuring database instances in public subnets in production, since it increases the risk of security attacks.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.77.0"

  name                 = "education"
  cidr                 = "10.0.0.0/16"
  azs                  = data.aws_availability_zones.available.names
  public_subnets       = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]
  enable_dns_hostnames = true
  enable_dns_support   = true
}

Subnet group

The next resource is an aws_db_subnet_group, which designates a collection of subnets that your RDS instance can be provisioned in. This subnet group uses the subnets created by the VPC module.

resource "aws_db_subnet_group" "education" {
  name       = "education"
  subnet_ids = module.vpc.public_subnets

  tags = {
    Name = "Education"
  }
}

This subnet group resource is an optional parameter in your aws_db_instance block below. Without it, Terraform creates your RDS instances in the default VPC.

Database instance

Review the aws_db_instance configuration.

resource "aws_db_instance" "education" {
  identifier             = "education"
  instance_class         = "db.t3.micro"
  allocated_storage      = 5
  engine                 = "postgres"
  engine_version         = "14.1"
  username               = "edu"
  password               = var.db_password
  db_subnet_group_name   = aws_db_subnet_group.education.name
  vpc_security_group_ids = [aws_security_group.rds.id]
  parameter_group_name   = aws_db_parameter_group.education.name
  publicly_accessible    = true
  skip_final_snapshot    = true
}

Note the following arguments.

username and password: The credentials for the root user.
publicly_accessible: Set to true for this tutorial's configuration. Use the default of false for production systems.
skip_final_snapshot: Set to true to disable taking a final backup when you destroy the database later in this tutorial.

You can review all of the supported arguments on the aws_db_instance resource documentation page.

Parameter group

Now review the definition for the aws_db_parameter_group.

resource "aws_db_parameter_group" "education" {
  name   = "education"
  family = "postgres14"

  parameter {
    name  = "log_connections"
    value = "1"
  }
}

This configuration enables connection logging for all instances using this parameter group. Note that the family parameter must correspond with the engine version of the RDS instance.

The parameter group resource contains all of the database-level settings for your RDS instance, which will be specific to the database engine and version you use.

Custom parameter groups are optional, and AWS will create the instance using a default parameter group if you do not supply one. However, you cannot modify the settings of a default parameter group, and changing the associated parameter group for an AWS instance always requires a reboot, so it is best to use a custom one to support modifications over the RDS life cycle.

Input variables

Note that the aws_db_instance root user password relies on an input variable. Open the variables.tf file to review its configuration.

variable "db_password" {
  description = "RDS root user password"
  type        = string
  sensitive   = true
}

Take note of the sensitive meta-argument for the db_password variable. This argument tells Terraform to hide the password from the output during Terraform operations. However, Terraform will store the password in plaintext in the state file.

Tip

To learn more about sensitive variables, check out the Protect Sensitive Input Variables tutorial.

Output variables

Now review the contents of the outputs.tf file.

output "rds_hostname" {
  description = "RDS instance hostname"
  value       = aws_db_instance.education.address
  sensitive   = true
}

output "rds_port" {
  description = "RDS instance port"
  value       = aws_db_instance.education.port
  sensitive   = true
}

output "rds_username" {
  description = "RDS instance root username"
  value       = aws_db_instance.education.username
  sensitive   = true
}

These outputs return details for the RDS instance that you will use to construct the database connection string later in this tutorial.

Provision resources

Now that you have reviewed the configuration, provision the RDS instance and associated resources.

First, set the db_password variable as an environment variable.

$ export TF_VAR_db_password="hashicorp"

Initialize the Terraform configuration.

$ terraform init
Initializing modules...
Downloading terraform-aws-modules/vpc/aws 2.77.0 for vpc...
- vpc in .terraform/modules/vpc

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v3.32.0...
- Installed hashicorp/aws v3.32.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Next, apply the configuration. Respond yes to the prompt to confirm.

$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:
##...
Plan: 14 to add, 0 to change, 0 to destroy.
##...
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
##..
aws_db_instance.education: Creation complete after 4m28s [id=education]

Apply complete! Resources: 14 added, 0 changed, 0 destroyed.

Outputs:

rds_hostname = <sensitive>
rds_port = <sensitive>
rds_username = <sensitive>

Terraform will now provision your resources. It may take 5-7 minutes for AWS to provision the RDS instance.

Verify your configuration by using the endpoint, the password, and username outputs to connect to the database using psql. Enter the password hashicorp when prompted.

$ psql -h $(terraform output -raw rds_hostname) -p $(terraform output -raw rds_port) -U $(terraform output -raw rds_username) postgres
Password for user edu:
psql (14.2, server 14.1)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

postgres=>

You are now connected to the database instance, verifying that the resource is provisioned as expected.

Create a new database called "hashicorp" within this instance.

$ CREATE DATABASE hashicorp;
CREATE DATABASE

Now verify that the hashicorp database is included in the list of databases in the instance by using the /list command. Along with the database you created, the list includes the default databases created in Postgres RDS instances.

$ \list
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 hashicorp | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres  | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 rdsadmin  | rdsadmin | UTF8     | en_US.UTF-8 | en_US.UTF-8 | rdsadmin=CTc/rdsadmin
 template0 | rdsadmin | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/rdsadmin          +
           |          |          |             |             | rdsadmin=CTc/rdsadmin
 template1 | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/edu               +
           |          |          |             |             | edu=CTc/edu
(5 rows)

You will use this database to verify replication later in this tutorial.

Leave the terminal by typing exit.

$ exit

Modify instance configuration

In addition to initially provisioning resources, you will likely need to modify the RDS configuration over the instance life cycle.

In main.tf, change the allocated storage from 5 to 10GB.

resource "aws_db_instance" "education" {
   name                   = "education"
   instance_class         = "db.t3.micro"
-  allocated_storage      = 5
+  allocated_storage      = 10
##...
}

Apply your changes. Respond to yes to the prompt to confirm.

$ terraform apply
Terraform will perform the following actions:

  # aws_db_instance.education will be updated in-place
  ~ resource "aws_db_instance" "education" {
      ~ allocated_storage                     = 5 -> 10
        id                                    = "education"
        name                                  = ""
        tags                                  = {}
        # (49 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
##...
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Outputs:

rds_hostname = <sensitive>
rds_port = <sensitive>
rds_username = <sensitive>

Even after the apply completes successfully, the change will still be pending. Verify by running terraform plan.

$ terraform plan
##...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_db_instance.education will be updated in-place
  ~ resource "aws_db_instance" "education" {
      ~ allocated_storage                     = 5 -> 10
        id                                    = "terraform-20210315135307744800000001"
        name                                  = "education"
        tags                                  = {}
        # (42 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Though some RDS configuration changes are safe to apply immediately, others (such as engine_version) require an instance reboot or may cause performance degradation (such as allocated_storage). By default, AWS will defer applying any changes that can cause degradation or outage until your next scheduled maintenance window. For a detailed breakdown of which attributes require a reboot, consult the AWS RDS documentation.

To make the changes take effect immediately, add the apply_immediately argument to aws_db_instance and set it to true.

resource "aws_db_instance" "education" {
   name                   = "education"
   instance_class         = "db.t3.micro"
   allocated_storage      = 10
+  apply_immediately      = true
##...
}

Apply your changes again. The proposed modifications will include the still-pending storage resize and the apply_immediately argument. Respond yes to the prompt to apply your changes.

$ terraform apply
##...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_db_instance.education will be updated in-place
  ~ resource "aws_db_instance" "education" {
      ~ allocated_storage                     = 5 -> 10
      + apply_immediately                     = true
        id                                    = "terraform-20210315135307744800000001"
        name                                  = "education"
        tags                                  = {}
        # (42 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
##...

This apply step will take longer than the previous one since Terraform will wait for the instance reboot to complete.

Once this update is complete, run terraform plan.

$ terraform plan
##...
No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

There are no pending changes remaining, confirming that Terraform resized the RDS instance. You can also verify this by navigating to your instance in the RDS console for your region and reviewing the database instances.

Warning

Use the apply_immediately argument with caution since it can allow unexpected instance reboots and downtime.

Provision a Read Replica

A read replica is one way to reduce load on the primary database. AWS will asynchronously copy all data from the primary database to the replica, to which you can then target all read queries.

Add the following configuration block to the main.tf file to declare a replica RDS instance.

resource "aws_db_instance" "education_replica" {
   name                   = "education-replica"
   identifier             = "education-replica"
   replicate_source_db    = aws_db_instance.education.identifier
   instance_class         = "db.t3.micro"
   apply_immediately      = true
   publicly_accessible    = true
   skip_final_snapshot    = true
   vpc_security_group_ids = [aws_security_group.rds.id]
   parameter_group_name   = aws_db_parameter_group.education.name
}

This provisions a read replica instance based on the aws_db_instance.education RDS instance. Since there is a replicate_source_db set, you don't need to set the required arguments for engine, allocated_storage, username, and password — Terraform will determine them from the corresponding values on the source RDS instance.

You will also need to enable backup retention on the primary instance to use it as a source database. Add the backup_retention_period argument to the primary instance configration.

resource "aws_db_instance" "education" {
    name                      = "education"
    instance_class            = "db.t3.micro"
    allocated_storage         = 10
+   backup_retention_period   = 1
##...
}

For the primary instance, you constructed the database connection string by passing the individual outputs to the psql parameters. You can also construct and output the entire connection string using Terraform's string interpolation functionality.

Add the output variable for the connection string for the new replica instance to the outputs.tf file.

output "rds_replica_connection_parameters" {
  description = "RDS replica instance connection parameters"
  value       = "-h ${aws_db_instance.education_replica.address} -p ${aws_db_instance.education_replica.port} -U ${aws_db_instance.education_replica.username} postgres"
}

Apply your changes to provision this additional instance. Respond yes to the prompt to confirm.

$ terraform apply
aws_db_parameter_group.education: Refreshing state... [id=education]
module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-03d07a04a25ae3f80]

##...

Plan: 1 to add, 1 to change, 0 to destroy.

Changes to Outputs:
  + rds_replica_connection_parameters = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

rds_hostname = <sensitive>
rds_port = <sensitive>
rds_replica_connection_parameters = "-h education-replica.cyfmek5yt2i5.us-east-2.rds.amazonaws.com -p 5432 -U edu postgres"
rds_username = <sensitive>

As with the original instance, it may take 5-7 minutes to provision the replica, and a few additional minutes to make updates to the primary instance.

Once it is complete, use the new endpoint to connect to the replica database instance to verify your configuration.

$ psql $(terraform output -raw rds_replica_connection_parameters)

Enter the password hashicorp when prompted.

Now, confirm that the replica instance has the database you created in the primary RDS instance. Use the /list command to see all of the databases.

Notice that the hashicorp database you created in the source RDS instance is in the database list for the replica instance, confirming that this instance is properly replicated from the primary.

$ \list
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 hashicorp | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres  | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 rdsadmin  | rdsadmin | UTF8     | en_US.UTF-8 | en_US.UTF-8 | rdsadmin=CTc/rdsadmin
 template0 | rdsadmin | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/rdsadmin          +
           |          |          |             |             | rdsadmin=CTc/rdsadmin
 template1 | edu      | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/edu               +
           |          |          |             |             | edu=CTc/edu
(5 rows)

Leave the terminal by typing exit.

$ exit

Clean up infrastructure

In this tutorial you provisioned and modified an RDS instance and read replica using Terraform. Clean up the infrastructure you have created. Respond yes to the prompt to confirm destroying the resources.

$ terraform destroy

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:
##...

Plan: 0 to add, 0 to change, 15 to destroy.

Changes to Outputs:
  - rds_hostname                      = (sensitive value)
  - rds_port                          = (sensitive value)
  - rds_replica_connection_parameters = "-h education-replica.cyfmek5yt2i5.us-east-2.rds.amazonaws.com -p 5432 -U edu postgres" -> null
  - rds_username                      = (sensitive value)

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

##...

Destroy complete! Resources: 15 destroyed.

Next steps

Terraform allows you to easily provision and manage AWS RDS instances using infrastructure as code. Since data storage resources are critical components of infrastructure, a declarative way to manage the resources over their life cycle will add an extra level of safety and consistency.

To learn more about managing RDS and other databases with HashiCorp tools:

Review the provider documentation for the aws_db_instance resource.
Check out the RDS module to learn more about configuration options of RDS and related resources.
Follow the tutorial for generating dynamic database credentials using HashiCorp Vault.

Cloudflare static website

EKS cluster