CloudTailor
Creating a new tool sometimes stems from a desire to address a repetitive, perhaps monotonous task. For me, this inspiration led to making CloudTailor.
I have found myself repeatedly setting up virtual machines (VMs) on Google Cloud Platform (GCP). The process looks the same: I configure VMs with Ubuntu, set them up with Docker to run containers configured with software images I want to run and then configure the VM on a custom domain with a Cloudflare Tunnel for SSL support. This process, while straightforward if familiar with all the tools, is time-consuming and error prone, especially when all one really wants to do is experiment with existing software and/or focus on deploying prototype code. Meh.
My goal was straightforward: develop a personal tool that could automate this process, making it more efficient and less tedious. I also wanted to learn how to use some OpenAI APIs. This led to the creation of a utility that helps automate the setup of a GCP VM with specific Docker images running SSL.
danielraffelCloudTailor uses a Bash script named config.sh, which serves as an interactive prompt to configure the required fields to get a VM up and running on GCP. This script begins by ensuring all necessary command-line tools are available and then guides users through various configuration stages. This includes prompting for inputs like the desired Google Cloud region, the type of server, operating system preferences, and details like their domain name, SSH public key path, Docker images (eg specific server software they want to install) and their OpenAI API key. The Bash script creates variables.txt, then triggers the execution of the Python script setup.py. This transitions to the next phase of setting up the Google Cloud VM.
The Python script setup.py manages the more complex tasks of setting up a server environment on GCP. It starts by importing user-defined variables from variables.txt.
The script's key tasks include interfacing with GCP by fetching the project ID and service account key, and managing the static IP address for consistent server access. It uses OpenAI's GPT-4 model to generate a docker-compose.yml file tailored to the user defined Docker images, crucial for orchestrating Docker container deployment. It creates a Terraform configuration file setup.tf to provision the GCP instance with detailed settings such as OS, server type, firewall rules and SSH configurations.
Additionally, setup.py generates several scripts and configuration files: setup_server.sh for setting up Docker on the VM, setup_cloudflare.sh for configuring Cloudflare Tunnel, updater.sh for updating Docker images and other software installed on the VM and docker-compose.service for Docker application lifecycle management (eg restarting Docker automatically when the VM is rebooted.)
A handy feature is the dynamic generation of Cloudflare Tunnel's config.yml file. The python script parses the docker-compose.yml file created by OpenAI to extract any Docker port mappings and then it creates corresponding Cloudflare Tunnel ingress entries, ensuring secure port exposure.
The Terraform file setup.tf is the final piece of the puzzle. It provisions a new GCP instance with the specified machine type and OS, and uploads essential files to the server to finish installing everything. The Terraform process ends by providing the user with the necessary IP address for SSH access. Although Terraform is capable of installing the server software automatically, this script prioritizes manual oversight to ensure these actions are monitored and managed. Once the user SSH's into their VM they run the scripts to setup their server with their software and are walked through configuring Cloudflare Tunnel on their DNS.
Presently, this is optimized for setting up Ubuntu servers, as the generated scripts specifically utilize Ubuntu's package installer apt-get. This was a fun project to make and I'll definitely be using it when I want to quickly spin up a customized e2 micro-instance VM. It's worth noting that this approach, while functional, is susceptible to various issues and is not recommended for production environments due to the potential fragility of the code it generates.