/AWS

How to Deploy MongoDB on AWS using Terraform: as a Service on ECS or on DocumentDB

In today’s cloud-first world, managing and deploying databases efficiently is crucial for maintaining high-performance applications. MongoDB, known for its flexibility and scalability, is a popular choice for developers and enterprises alike. When combined with the power of AWS, you can leverage robust infrastructure and services to ensure your MongoDB deployments are secure, scalable, and cost-effective.

This blog post will guide you through deploying MongoDB on AWS using Terraform, an Infrastructure as Code (IaC) tool that allows you to manage and automate your cloud resources. Whether you’re looking to run MongoDB as a service on Amazon Elastic Container Service (ECS) or leverage Amazon DocumentDB (a fully managed, MongoDB-compatible database service), this tutorial has you covered.

By the end of this guide, you’ll have a step-by-step walkthrough to get your MongoDB deployment up and running on AWS. Whether you’re a DevOps engineer, a cloud architect, or a developer looking to automate your infrastructure, this blog post will equip you with the knowledge to deploy MongoDB efficiently on AWS using Terraform.

The whole code for this solution is available on Github Here. If you want to follow along, this can be used to stand up your own infrastructure.

Pre Requisites

Before I start the walkthrough, there are some some pre-requisites which are good to have if you want to follow along or want to try this on your own:

  • Basic AWS knowledge
  • Github account to follow along with Github actions
  • An AWS account
  • AWS CLI installed and configured
  • Basic Terraform knowledge
  • Terraform Cloud free account. You can sign up here. Its not mandatory but I have used this for state management in the Terraform scripts. You can use S3 as well.

With that out of the way, lets dive into the details.

What is MongoDB?

MongoDB is a popular NoSQL database that provides a flexible and scalable solution for storing and managing data. It is known for its document-oriented data model, which allows you to store data in JSON-like documents. MongoDB is widely used in modern web applications, mobile apps, and other data-intensive use cases.
MongoDB is designed to be highly scalable and fault-tolerant, making it a great choice for applications that require high availability and performance. It supports features like sharding, replication, and auto-scaling, which help you scale your database as your application grows.

What is Amazon DocumentDB?

Amazon DocumentDB is a fully managed, MongoDB-compatible database service that provides high performance, scalability, and availability. It is designed to be compatible with existing MongoDB applications, so you can easily migrate your workloads to Amazon DocumentDB without making any code changes.
Amazon DocumentDB is built on a distributed, fault-tolerant architecture that automatically handles tasks like backups, patching, and scaling. It provides features like read replicas, point-in-time recovery, and encryption at rest to help you secure and manage your data effectively.

What is ECS?

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that allows you to run and scale containerized applications on AWS. ECS simplifies the process of deploying, managing, and scaling containers by providing features like service discovery, load balancing, and auto-scaling.
With ECS, you can run containers on a cluster of EC2 instances or Fargate, a serverless compute engine for containers. ECS integrates with other AWS services like IAM, CloudWatch, and VPC to provide a secure and scalable environment for your containerized applications.

About the Solution

Now lets understand the solution that we are going to implement. We will be deploying MongoDB on AWS using Terraform in two different ways:

Deploy MongoDB as a service on Amazon Elastic Container Service (ECS)
Below image shows the architecture of the solution.

mongoecs

  • Lambda: This is a Lambda function which we will deploy to test the connectivity to the Mongo DB service and perform some test operations like creating a document. I am using CDK to deploy the Lambda function. The Lambda function is deployed inside the same VPC as the ECS service so that it can connect to the MongoDB service. 

    const testLambda = new lambda.DockerImageFunction(this, 'test-lambda', {
  functionName: 'test-lambda',
  code: lambda.DockerImageCode.fromImageAsset('assets/lambda_code'),
  role: lambdaRole,
  memorySize: 256,
  timeout: cdk.Duration.seconds(30),
  vpc: vpc,
  vpcSubnets: {
    subnetType: cdk.aws_ec2.SubnetType.PRIVATE_WITH_EGRESS
  },
  securityGroups: [sg],
  environment: {
    'MONGO_USER': 'admin',
    'MONGO_PWD_SSM': '/mongodb/MONGO_INITDB_ROOT_PASSWORD'
  }
});
  • ECS Service Registered to Service Discovery: The ECS service which is deployed to the ECS cluster, is registered to a Cloudmap namespace. This ensures th service can be reached using an user friendly domain name 
resource "aws_service_discovery_private_dns_namespace" "mongo_dns" {
  name = "mongo.dns"
  vpc  = var.vpc_id
}

resource "aws_service_discovery_service" "mongo_discovery" {
  name = "mongodb"
  dns_config {
    namespace_id = aws_service_discovery_private_dns_namespace.mongo_dns.id
    dns_records {
      ttl  = 10
      type = "A"
    }
  }
  health_check_custom_config {
    failure_threshold = 1
  }
}
  • ECS Cluster: The ECS cluster where the MongoDB service is deployed.
  • Networking: Various networking components are deployed to support the deployment and connectivity. Some of the components are VPC, Subnets, Security Groups, Route Tables etc.
  • ECS Service: The ECS service which is deployed to the ECS cluster. This service is the MongoDB service which is running in a container. The service runs tasks in a private subnet to increase the security. The task definition for the task defines the image and the initial credentials for the MongoDB. 
container_definitions = jsonencode([
    {
      name      = "mongo",
      image     = "public.ecr.aws/docker/library/mongo:8.0-rc",
      cpu       = 256,
      memory    = 512,
      essential = true,
      portMappings = [
        {
          protocol      = "tcp"
          containerPort = 27017
          hostPort      = 27017
        }
      ]
      mountPoints = [
        {
          sourceVolume  = "sharedvol"
          containerPath = "/data/db"
          readOnly      = false
        },
      ],
      environment = [
        {
          name  = "MONGO_INITDB_ROOT_USERNAME"
          value = "admin"
        },
        {
          name  = "MONGO_INITDB_DATABASE"
          value = "mongodb"
        },
      ],
      secrets = [
        {
          name      = "MONGO_INITDB_ROOT_PASSWORD"
          valueFrom = var.mongo_ssm_param_name
        }
      ]  
    }
  • EFS for Mongo Storage: An EFS volume is created to store the data for the MongoDB. This is mounted to the ECS task. This ensures the data is persistent even if the container is stopped. 
volume {
  name = "sharedvol"

  efs_volume_configuration {
    file_system_id     = aws_efs_file_system.mongo_fs.id
    transit_encryption = "ENABLED"
    authorization_config {
      iam = "ENABLED"
    }
  }
}

Deploy MongoDB on Amazon DocumentDB
Now lets see the architecture of the solution when we deploy MongoDB on Amazon DocumentDB. Below image shows the architecture of the solution.

mongodocdb

  • Lambda: This is a Lambda function which we will deploy to test the connectivity to the Mongo DB service and perform some test operations like creating a document. This Lambda is deployed inside the same VPC as the DocumentDB so that it can connect to the DocumentDB. I am using CDK to deploy the Lambda function. The lambda code is part of the assets in the CDK project. 
const docdbtestLambda = new lambda.DockerImageFunction(this, 'docdb-test-lambda', {
      functionName: 'docdb-test-lambda',
      code: lambda.DockerImageCode.fromImageAsset('assets/docdb_lambda'),
      role: lambdaRole,
      memorySize: 256,
      timeout: cdk.Duration.seconds(30),
      vpc: vpc,
      vpcSubnets: {
        subnetType: cdk.aws_ec2.SubnetType.PRIVATE_WITH_EGRESS
      },
      securityGroups: [sg],
      environment: {
        'MONGO_ENDPOINT': 'mongodb://mongoadmin:mongopassword@<cluster_endpoint>>:27017/testdb?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false'
      }
    });
  • DocumentDB Cluster: The DocumentDB cluster which is deployed. The cluster is deployed in a private subnet to increase the security. The cluster is deployed with one instance. For better availability more instances can be added. The cluster is deployed with a custom parameter group to enable the MongoDB compatibility. 
resource "aws_docdb_cluster" "mongo_cluster" {
  skip_final_snapshot     = true
  db_subnet_group_name    = "${aws_docdb_subnet_group.mongo_grp.name}"
  cluster_identifier      = "mongo-app-cluster"
  engine                  = "docdb"
  master_username         = var.mongo_username
  master_password         = "${var.docdb_password}"
  db_cluster_parameter_group_name = "${aws_docdb_cluster_parameter_group.mongo_params.name}"
  vpc_security_group_ids = [var.mongo_sg_id]
}
  • Networking: Various networking components are deployed to support the deployment and connectivity. Some of the components are VPC, Subnets, Security Groups, Route Tables etc.

Next we will see how we deploy the stacks.

Deploy MongoDB on ECS using Terraform

First lets see how we deploy MongoDB on ECS using Terraform. If you are following along, you can clone the repository to start. Before starting there are few pre-requisites which you need to have:

  • AWS CLI installed and configured
  • Terraform installed
  • Terraform cloud account. You can sign up here. Its not mandatory but I have used this for state management in the Terraform scripts. You can use S3 as well.
  • Docker installed
  • An AWS account

Folder Structure

To follow along, you will have to understand the folder structure of the repository. Below is the folder structure of the repository.

mongofolder

  • .github: This folder contains the Github actions workflows which are used to deploy the stacks.
  • mongo-documentdb: This folder contains the Terraform scripts to deploy MongoDB on DocumentDB.
  • mongo-ecs: This folder contains the Terraform scripts to deploy MongoDB on ECS.
  • test_lambda: This folder contains CDK project to deploy the test Lambda functions. These Lambda functions will be used to test the connectivity to the MongoDB services.

Deployment Pipeline

Now lets see how we deploy the MongoDB stack. I am using Github actions to automate the deployment. If you are following along, there are few pre-requisites which you need to have:

  • Github account
  • AWS account
  • Terraform Cloud account. You can sign up here. Its not mandatory but I have used this for state management in the Terraform scripts. You can use S3 as well.

Before starting the deployment, create an API token on Terraform cloud and put the token as a secret on Github. This secret will be used by the Actions workflow to connect to Terraform cloud and deploy.

mongosecret

Now lets see the Github actions workflow which is used to deploy the MongoDB stack. Below image shows the workflow steps.

mongoworkflow

The workflow has the following steps:

  • The workflow is triggered manually from the Github actions tab on the repo

  • First we setup the Terraform to be able to connect to Terraform cloud. We pass the API token secret to perform the setup 

    name: Setup Terraform
uses: hashicorp/setup-terraform@v1
with:
cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

  • Next we run the Terraform init to initialize the Terraform scripts 

    name: Terraform Init
  id: init
  run: |
    cd mongo-ecs/infrastructure
    terraform init -input=false

  • Next we run the Terraform plan to see the changes which will be applied 

    name: Terraform Plan
  id: plan
  run: |
    cd mongo-ecs/infrastructure
    terraform plan -no-color -input=false
  continue-on-error: true  
name: Terraform Plan Status
  if: steps.plan.outcome == 'failure'
  run: exit 1

  • Next we run the Terraform apply to apply the changes 

    name: Terraform Apply
  run: |
    cd mongo-ecs/infrastructure
    terraform apply --auto-approve -input=false

Now that we saw how the workflow works, lets deploy this!!!

Testing the Deployment

To start the deployment, make sure the code is pushed to the Github repo. 

git add .
git commit -m "Initial commit"
git push origin main

Before starting the deployment, make sure to set a desired password in the ssm param. This ssm param will be read by the ECS service to set the initial Mongo password. In a Production grade system this should be a proper protected secret and not put in a plain text as this example. For this blog post I am simplifying the storage. 

  resource "aws_ssm_parameter" "mongo_password" {
  name  = "/mongodb/MONGO_INITDB_ROOT_PASSWORD"
  type  = "SecureString"
  value = "dummy"

  lifecycle {
    ignore_changes = [value]
  }
}

Now navigate to the Github repo, and navigate to the Actions tab. Navigate to the workflow named “Deploy Infra for Mongo on ECS”. Click on the workflow and click on the “Run workflow” button. It should start the workflow. Wait for it to finish. You can check the logs to see progress.

mongoworkflowrun

Lets check on AWS what was deployed. Login to AWS console and navigate to the corresponding services.

ECS Service for MongoDB
mongoevsservice

DNS Service for MongoDB
mongodns

EFS Volume for MongoDB
mongoefs

All the services are deployed and the Mongo Service is up and running. Now lets test the connectivity to the MongoDB service. I have included a test Lambda in my repo. If you are following along, to deploy the test lambda for testing ECS-Mongo, run the workflow named “Deploy Testing Lambda”. This will deploy the test Lambda function. Before deploying make sure to set the access key and secret key as secrets in the Github repo.

mongotestlambda

The Lambda is defined in a CDK stack. The mongo credentials are passed as environment variables to the Lambda. The Lambda code is part of the assets in the CDK project. 

const testLambda = new lambda.DockerImageFunction(this, 'test-lambda', {
      functionName: 'test-lambda',
      code: lambda.DockerImageCode.fromImageAsset('assets/lambda_code'),
      role: lambdaRole,
      memorySize: 256,
      timeout: cdk.Duration.seconds(30),
      vpc: vpc,
      vpcSubnets: {
        subnetType: cdk.aws_ec2.SubnetType.PRIVATE_WITH_EGRESS
      },
      securityGroups: [sg],
      environment: {
        'MONGO_USER': 'admin',
        'MONGO_PWD_SSM': '/mongodb/MONGO_INITDB_ROOT_PASSWORD'
      }
    });

Lets run the workflow to deploy the Lambda

mongotestlambdarun

The Lambda should be deployed. Lets check the Lambda and its config

mongotestlambdaview

We have passed the SSM param name as an environment variable to the Lambda. This is the password for the MongoDB. The Lambda will use this password to connect to the MongoDB service.

Now lets test the connectivity to the MongoDB service. Navigate to the Lambda and click on the “Test” button. The Lambda should connect to the MongoDB and log some messages showing it connected and created some sample Document

mongotestlambdatest

The lambda performs these steps to test the connectivity:

  • Connect to the MongoDB service
  • Creates a test DB
  • Creates a test collection named “users”
  • Inserts a document in the collection as an user row

All of the steps can be seen from the logs above. So we have successfully connected to the Mongo Service from the Lambda. Now lets move on to deploying MongoDB on DocumentDB.

Deploy MongoDB on DocumentDB using Terraform

Now lets deploy MongoDB on DocumentDB using Terraform. If you are following along, you can clone the repository to start. Before starting there are few pre-requisites which you need to have:

  • AWS CLI installed and configured
  • Terraform installed
  • Terraform cloud account. You can sign up here. Its not mandatory but I have used this for state management in the Terraform scripts. You can use S3 as well.
  • Docker installed
  • An AWS account

Folder Structure

The folder structure for the DocumentDB deployment is similar to the ECS deployment. The infrastructure for DocDB is put into a separate folder.

  • mongo-documentdb: This folder contains the Terraform scripts to deploy MongoDB on DocumentDB.
  • test_lambda: This folder contains CDK project to deploy the test Lambda functions.
    mongodocdbfolder

Deployment Pipeline

The pipeline flow is same as the ECS deployment Only difference is that it deploys the infrastructure from a different folder. The workflow for this deployment is defined in ‘deploy-mongo-docdb.yml’.
Follow the same pre-requisites from the ECS deployment.

mongoworkflow

Now lets deploy this!!!

Testing the Deployment

Before you start the deployment, update the mongo admin passwords in the mongo-documentdb/infrastructure/main.tf 

mongo_username = "mongoadmin"
docdb_password = "mongopassword"

In an actual deployment, these passwords need to come from a secure secret store and shouldn’t be ut as a plain text. For this example I am putting the credentials in plain text.

Now navigate to the Github repo, and navigate to the Actions tab. Navigate to the workflow named “Deploy Infra for Mongo on DocumentDB”. Click on the workflow and click on the “Run workflow” button. It should start the workflow. Wait for it to finish. You can check the logs to see progress.

mongoworkflowrun1

Lets check on AWS what was deployed. Login to AWS console and navigate to the corresponding services.

DocumentDB Cluster for MongoDB
mongodocdbcluster

All other components are same as the ECS deployment as we are using the same networking. The DocumentDB cluster is deployed and the Mongo Service is up and running. Now lets test the connectivity. We will deploy a test Lambda. The CDK code for the test Lambda is in the repo in assets called ‘docdb_lambda’.

mongodocdbfolder

Lets get the Mongo endpoint. Navigate to the Cluster details page and copy the endpoint

mongodocdbendpoint

Before we can deploy, update the mongo endpoint environment variable in the CDK code. This is the endpoint of the DocumentDB cluster. 

const docdbtestLambda = new lambda.DockerImageFunction(this, 'docdb-test-lambda', {
      functionName: 'docdb-test-lambda',
      code: lambda.DockerImageCode.fromImageAsset('assets/docdb_lambda'),
      role: lambdaRole,
      memorySize: 256,
      timeout: cdk.Duration.seconds(30),
      vpc: vpc,
      vpcSubnets: {
        subnetType: cdk.aws_ec2.SubnetType.PRIVATE_WITH_EGRESS
      },
      securityGroups: [sg],
      environment: {
        'MONGO_ENDPOINT': 'mongodb://<mongo_user>:<mongo_password>>@<cluster_endpoint>>:27017/testdb?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false'
      }
    });

The Lambda is deployed to the same VPC as the DocDB cluster. This ensures the Lambda can connect to the DocDB cluster.

Push the changes to the repo. 

git add .
git commit -m "Updated the endpoint"
git push origin main

Now navigate to the Github repo and run the workflow named “Deploy Testing Lambda”. This will deploy the test Lambda function. Before deploying make sure to set the access key and secret key as secrets in the Github repo.

mongodocdbtestlambdarun11
mongodocdbtestlambdarun22

Wait for the workflow to finish. The Lambda should be deployed. Lets check the Lambda and its config

mongodocdbtestlambdaview

The mongo endpoint is set as the environment variable.

Now lets test the Lambda. Navigate to the Test tab and click on “Test”. The Lambda should connect to the DocumentDB and log some messages showing it connected and created some sample Document.

mongodocdbtestlambdatest

The logs show that the Lambda connected to the DocumentDB and created a test document. This means the connectivity is successful.

mongodocdbtestlambdatest1

Great!!! We have successfully deployed MongoDB on DocumentDB as well as on ECS. We have tested the connectivity to the MongoDB service from a Lambda. This is a great way to automate the deployment of MongoDB on AWS using Terraform. Remember to delete the stacks after you are done to avoid any charges.

Conclusion

In this blog post, we explored how to deploy MongoDB on AWS using Terraform. We covered two deployment scenarios: running MongoDB as a service on Amazon Elastic Container Service (ECS) and leveraging Amazon DocumentDB, a fully managed, MongoDB-compatible database service.
By following the step-by-step walkthrough, you learned how to automate your MongoDB deployments on AWS, ensuring they are secure, scalable, and cost-effective.
If any issues or any queries, feel free to reach out to me from the Contact page.

Amlan

Amlan

Cloud architect and DevOps engineer. Love to code and develop new stuff. A nerd by nature.

Read More