The Thought:

In the fast-paced world of software development, staying ahead often means automating repetitive tasks. One such task that has become critical is the deployment of applications. Services like Render.com have gained popularity for simplifying the deployment process. But what if you want to build your own deployment system from the ground up? It was precisely this question that struck me one evening at 9 PM, igniting my determination to replicate a key feature of Render.com – deploying applications automatically with each new pull request. Armed with a trusty whiteboard and pen, I embarked on this exciting journey.

The Problem

Before diving into the technical details, let’s understand the problem we’re tackling. Our goal is to create a system that, like Render.com, deploys applications automatically when new pull requests are made. This means every time a developer initiates a pull request, a new instance of the application is spun up. To achieve this, we must break down the problem into several subproblems:

Creating DNS Records:

• We need to set up a wildcard DNS record pointing to our server. This record should capture all subdomains, ensuring that new pull requests can be accessed via unique subdomains.

Configuring GitHub Webhooks:

• GitHub webhooks will be our bridge between the repository and our server. When a new pull request is raised, the webhook will transmit data to our server for processing.

Server Configuration and Scripting:

• We’ll need to configure our server to listen for incoming webhook data.

• We’ll create a script that activates when a new pull request is received, automating various tasks such as cloning the repository, generating subdomains, setting up Nginx templates, and linking the templates to the appropriate repository folders.

Application Deployment:

• Finally, our system should deploy the application associated with the pull request on a unique subdomain, handling SSL setup, web domain configuration, and other necessary tasks.

In the following sections, we’ll dive deeper into each of these subproblems, outlining the steps and solutions for building this automated deployment system from scratch.

Setting Up DNS Records

To kickstart our automated deployment system, we must ensure that we can dynamically create subdomains for each new pull request. This begins with setting up wildcard DNS records.

I purchased a domain from GoDaddy and utilized Cloudflare for DNS management. The crucial step was adding a wildcard DNS entry. We set the ‘*’ as the value and pointed it to our server’s IP address. This wildcard DNS entry captures all subdomains, making it possible for us to access new pull requests via unique subdomains. To verify that everything was working as expected, I tried accessing a subdomain like ‘test.domain.com,’ even though no specific DNS entry existed for it. The result? Success! This was a critical first step in our journey. The browser should display page according to your server configurations.

Configuring GitHub Webhooks

Our next move was to set up GitHub webhooks, the communication link between our repository and the server. Here’s how you can do it:

1. In your GitHub repository’s settings, navigate to ‘Webhooks’ in the sidebar.

2. Create a new webhook with an appropriate name, and point it to your server and make sure to check pull request box while configuring.

After creating the webhook, it will attempt to send data to our server but, of course, will fail, as we haven’t yet configured our server to receive webhook events. This sets the stage for our next task.

Server Configuration and Scripting

Configuring the server is a crucial aspect of our deployment system. We need it to listen for incoming webhook data and execute specific scripts when new pull requests are received.

Here’s an example of the configuration JSON for the webhook on your server:

[
    {
        "id": "github-webhook",
        "execute-command": "./webhookscript.sh",
        "command-working-directory": ".",
        "response-message": "Webhook received and processing...",
        "pass-arguments-to-command": [
            {
                "source": "payload",
                "name": "action"
            },
            {
                "source": "entire-payload"
            }
        ]
    }
]

The webhookscript.sh script is where the magic happens. This script processes the data received from GitHub webhooks and automates the deployment of new pull requests. It checks for the ‘action’ in the payload, and if it’s ‘opened,’ the script performs a series of actions:

Application Deployment

The final piece of the puzzle is application deployment. With the server and script configured to handle incoming pull requests, we’re ready to deploy applications automatically. The script performs the following actions:

• Generates a unique subdomain based on the pull request title.

• Sets up Nginx templates for serving the new subdomain.

• Clones the repository, switches to the pull request branch, and deploys the application.

• Handles SSL setup and web domain configuration for added security.

#!/bin/bash
if [ "$1" == "opened" ]; then
    action=$(echo "$2" | jq -r '.action')
    pull_request_title=$(echo "$2" | jq -r '.pull_request.title')
    pull_request_url=$(echo "$2" | jq -r '.pull_request.html_url')
    sender_login=$(echo "$2" | jq -r '.sender.login')
    clone_url=$(echo "$2" | jq -r '.repository.ssh_url')

    if [ -z "$action" ] || [ -z "$pull_request_title" ] || [ -z "$pull_request_url" ] || [ -z "$sender_login" ]; then
        echo "Error: Required fields are missing or empty in the payload."
        exit 1
    fi
    prname="${pull_request_title// /}"
    prname="${prname}.domain.com"
    alias="www.$prname"
    v-add-web-domain server $prname ipv4 yes $alias
    v-add-letsencrypt-domain server $prname $alias no
    v-add-web-domain-ssl server $prname /home/server/conf/web/$prname
    v-add-web-domain-ssl-force server $prname
    web_directory="/home/server/web/$prname/public_html"
    cd $web_directory

    if [ ! -d "$web_directory/repo_name" ]; then
        git clone "$clone_url"
    fi
    cd repo_name/app
    cp -r ~/.env $web_directory
    npm install
    npm run build
    mv -vf dist/* ../../
fi

This comprehensive system ensures that each new pull request triggers the creation of a dedicated subdomain, where the application can be accessed and tested. It’s a powerful way to streamline your development workflow.

The Adventure:

In this blog post, we’ve taken a deep dive into the world of automated application deployment with GitHub webhooks. We’ve broken down the problem into manageable subproblems, from setting up DNS records to configuring webhooks, scripting, and application deployment. By building your own deployment system, you have the power to automate a critical part of your development workflow and make the process more efficient.

Please ensure that you replace variables such as ‘server,’ ‘ipv4,’ ‘domain.com,’ and ‘repo_name’ with their respective values. In our VPS setup, we utilized Hestia CP. The commands, such as ‘v-add-web-domain’ or those beginning with the ‘v-‘ prefix, pertain to Hestia Control Panel, so be sure to apply them in accordance with your own system configurations.

As I wrapped up my work around 1 am, I couldn’t help but reflect on the fact that coding often feels like caffeine, keeping you awake throughout the night.

Wishing you productive coding and seamless deployment!