DR and Dev Environment architecture for a Wordpress Blog

Being a techie husband of a non-techie wife, I have to face a lot of Tech questions for her. She is a fashion blogger and I am in charge of managing the technical stuff for the blog.She once asked me ’What will happen if my blog goes down?’. That triggered me to think about building some kind of backup for the blog. Coincidentally at the same time I was preparing for my AWS certifications. So this thought got me excited and inspired me to build something in AWS.

In this write-up I will explain in high level, how I built a DR environment and a Development environment for my wife’s blog.

High Level Architecture

Lets start with a high level architecture the set up. Below is the architecture which was built in AWS to handle the DR environment and the Dev environment functionalities.

DR Environment details
The standby environment is built on AWS Opsworks stack. Custom recipes are being used to bootstrap instances in different layers. A lambda function handles failover DNS changes when primary blog is down or facing some issues. This stack is kept in sync using other lambda functions which we will discuss later. We will go into more details of this architecture in upcoming sections.

Dev Environment Details
Dev environment is launched whenever there is a need to make any changes to the blog. The environment is launched using a cloudfomrtaion template and data(including metadata) copied from the standby DR environment.

Monitoring
This full process and the componenets are being monitored using Cloudwatch. I have setup multiple events and Log groups to capture and monitor the componenets of the environment.

Details of DR Architecture

Lets go into details of the architecture for DR environment. Below is a high level look of the architecture.

The architecture consists of below components and processes:

Networking
It consists of a Route 53 service which provide a single endpoint for end users to access the backup blog. Route 53 routes requests to an Elastic Load balancer. These are housed inside a VPC created for the components of the DR environment. The VPC contains Public and Private subnets spanning multiple AZs inside region(us-east-1). The public subnets contain the Web server part of the architecture. All the components in the public subnets add up to server the blog to the ELB. The private subnets house the MySQL data base and the multi-az failover backup.

Opsworks Stack
The Opsworks Stack collects the whole architecture in a single stack. There are two layers which build the stack and complete the architecture:

• Web Layer: This layer serves up the blog to the ELB. This layer consist of multiple EC2 instances and is auto scaling enabled to scale in or out according to load. It is configured to used load based scaling. To make sure that all the launched instances share the same WP content folder, all the instances share an EFS storage which is mounted as volumes in each of the instances. To support high availability, the instances are spread across 2 AZs. Each of the instances are bootstrapped using a custom CHEF recipe which is handled by Opsworks. The instances itself are running Wordpress Docker containers. The WP folder from the EFS storage is mounted as a volume on the docker containers. Below is a high level view of what the Docker container and the CHEF recipe consist of.

• DB Layer: This layer consists of adding an existing RDS MySQL database to the stack. The DB is being created and kept up to date with a process I will discuss later. The Database is deployed in a multi-AZ failover architecture where a failover DB is housed in another AZ and being kept in sync. The DB is provisioned the subnet group consisting of Private subnets only accessible from instances in the Web layer.

Sync Process
Its easy to design and launch the components of the architecture. Main challenge is to keep the WP folder(configuration, Themes, styles etc.) and the database in sync with the primary blog. I have used multiple scripts and job to achieve the sync between primary blog and the fail-over blog. Below is a high level view of the process I built to perform this sync. This process runs nightly to fetch any changes in the primary blog to the DR blog. For days when there are heavy changes or work done in the primary blog, I run the process manually couple of times a day to make sure all changes are synced properly.

The process starts with two CRON jobs on the primary blog server. I have two scripts which reads the DB and the WP folder and uploads to a S3 location on my AWS account. The upload triggers a lambda function which handles updating of the WP contents and the DB contents to respective EFS storage and the RDS database. The RDS MySQL is configured to take a snapshot everyday and keep it as a backup. The overall process is being monitored using Cloudwatch and I have alarms setup to notify me of any errors.

Fail-Over Process
The fail-over to DR is also automated using a lambda function. This process can be well explained by the below graphic.

Details of Dev Architecture and Process

Now that we have the fail-over architecture ready, we should have a process to perform, test and deploy new changes to the primary blog. Building on the similar components from the DR architecture, I have established an identical environment but not as highly available or fault tolerant as the DR. Because its Dev after all. I also follow a process to deploy new changes to the primary blog. Below is the high level architecture of the Dev environment.

The same custom CHEF cookbooks and Dockerfiles are used to prepare the EC2 instance for development. This environment is launched everytime a new change is to be deployed and once done, this environment is destroyed. Below is an overall view of how the launch is handled and different components of the process.

Dev Environment Components
To launch the Dev environment, I have built various different components which are executed in a controlled way using a Jenkins pipeline.The components involved in launching the Dev infrastructure consist of below parts:

• Cloudformation Template: This handles the overall provision of various portions of the architecture like newtwork, DB etc.
• Docker File: To run the Wordpress container on the instance
• CHEF Cookbook: This is the same custom cookbook containing the recipe to bootstrap the instance and get the blog running

Below image gives an overall picture of what these files contain

Dev Launch Flow
I have implemented the below flow in a Jenkins pipeline job to control the launch of the Dev architecture componenets. The flow consist of two phases:

• Launch Phase: This phase handles launching of all components of the architecture like network, DB etc.
• Bootstrap Phase: This phase handles installing required packages on the instance. The custom CHEF recipe handles the bootstrapping and finishes with getting the Wordpress docker container up and running.

More details about the flow below

Details of Development process

For the development flow, I havent yet automated all the steps and still working on details on how to acieve a full automation. But in its current state, below image can describe the overall flow I follow.

The flow consists of below phases:

• Launch Phase: This phase handles launching and preparing the Dev environment components
• Development Phase: This is the actual development phase where I make changes to the blog in the Dev environment.
• Testing Phase: In this phase I launch a testing environment and test the changes.
• Deploy Phase: This is the final phase where I deploy the changes to the primary blog and destroy the Dev environment.

Epilogue

This brings me to the end of this long process which I built. Hope I was able to cover all the portions of the process and if something is missing please let me know and I will try to explain. When I developed this process, I was still learning these various technologies. So going back through some of these details, I see there are lot of romm for improvements. I have started working on some improvements and hopefully will be able to post a modified and improved process sometime soon. Of course, I am also looking for suggestions from the community too.

I am also working on making the actual code files more presentable so I can post the actual files as an update to this post. That will complete this long description.

Any suggestions, questions, issues please email me at : amlanc@achakladar.com .