Learning About Google Cloud By Developing a Ghost.org Installer
If you've ever wanted to self-host Ghost.org on Google Cloud without the hassle, you're in luck. I recently embarked on learning more about Google's Cloud and decided to automate this process, using the Google Cloud Command Line Interface (CLI) and various Google services. Google Cloud has a free-tier E2-Micro instance server ideal for running low traffic websites or blogs. Why not make it accessible to more people?
Here’s how I made it work and some things I learned. If you're just interested in the code the GitHub project is called gcloud_ghost_instancer.
danielraffelThe Script's Workflow
Starting Locally
The script initiates on your local machine and collects essential parameters like the URL where you intend to host your blog, an option to customize your Virtual Machine (VM) name and an option to setup your Mailgun account if you want Ghost to send emails to subscribers. It then auto-generates secure passwords to associate with your VM's root, service-account and mysql user, assembles custom install parameters for Ghost, and stores all of this sensitive data as secrets that the VM will have access to via Google Secret Manager.
Spinning Up the VM
Next, the script launches a VM in Google Cloud, applying custom firewall rules and running a setup script that configures user/passwords and sets permissions. It also instructs the VM to download and run a GitHub script that sets up the the Ghost software. Some of the software installed on the VM requires access to passwords that were generated locally, these are securely fetched from Google Secret Manager using the pre-installed Google Cloud CLI.
Back to Local
Once Ghost is fully installed on the VM, the script transitions back to your local machine. To install the software I found that I needed a mid-tier machine. So, after the install the script downgrades the VM from an E2-Medium to a free-tier E2-Micro instance, assigns a static IP, and SSH's into it to verify that Ghost is running.
Overcoming Challenges
Checking for Google Cloud CLI
First off, I had to ensure the user had Google Cloud CLI installed. The check and installation were straightforward, and I even found a feature to disable setup prompts to keep the script running smoothly. Since this was a personal learning exercise I decided I was only going to optimize for macOS. Sorry!
Regional Limitations
Free-tier E2-Micro servers are available in specific locations. To account for this, I added a simple picker to the script, allowing the user to choose from available regions.
Defining a Naming Strategy
To minimize the risk of naming conflicts across various components, I adopted a consistent naming strategy based on the VM instance name a user selected. This allowed for easy identification and helped avoid accidental overlaps. Here are some examples:
- Service account password:
service-account-password-$INSTANCE_NAME - Root password:
root-password-$INSTANCE_NAME - Public keys for root and service accounts:
root_key-${INSTANCE_NAME}.pubandservice-account_key-${INSTANCE_NAME}.pub - Generic secret names:
mysql-password-$INSTANCE_NAME
Conforming to Existing Conventions
The script includes a feature to customize the VM's name while checking for naming conflicts and adhering to Google Cloud's naming conventions.
The Intricacies of SSH and TTY Allocation
While using SSH to access the VM, I learned about forcing pseudo-terminal allocation with the -tt option. In theory, this should allow tty allocation even when SSH has no local tty, which is useful for running commands with stdin redirection on the remote machine. I hoped this would solve the issue of running the Ghost installer in an SSH window and accepting input. Unfortunately, it didn't work as anticipated. This led me to the workaround of collecting and passing parameters to the installer manually—a clear case where theory and practice diverged, but offered a valuable learning experience nonetheless.
Handling Blog and Mailgun Configurations
My initial plan was to run Ghost's interactive installer within an SSH window to configure the Blog URL and Mailgun settings. However, as I mentioned part of this approach failed due to issues with the way I was running Ghost Install. As a workaround, I developed questions in the script to collect settings I would need to provide to the Ghost CLI beforehand. I then stored these parameters in Google Secret Manager, which allowed me to securely pass them to the Ghost installer at runtime on the server.
Variables and Memory
When running the script locally, variables were in memory but got cleared once I SSH'd into the VM. To access them again post-exit, I stored non-sensitive data on disk for future retrieval.
Dealing with Memory Limitations
While attempting to run my installer on an E2-Micro (1GB) instance, I encountered frequent timeouts and maxed-out memory. Even upgrading to an E2-Small (2GB) didn't solve the issue. Ultimately, I opted for an E2-Medium (4GB) instance to ensure stability for setup. And, downgraded to an E2-Micro post setup. This incurred a nominal one-time cost. To optimize memory on the E2-Micro, I disabled snapd. This is a useful tweak for a resource-limited free-tier instance. Although I thought about using Swap memory, I found it wouldn't provide the extra memory when my installer most needed it.
Functionality vs. Optimization
I considered switching from Node.js to a compatible package manager like Bun to reduce memory consumption. However, I decided to prioritize getting the script to work reliably on an E2-Medium over optimizations. I had a hunch there might be Node.js compatibility issues with Bun and didn't want to go down that rabbit hole.
Considering Password Management and Setup Parameters in Google Cloud
Reusable Function for Password Generation
Managing passwords securely is paramount. I created a reusable function generate_and_store_password() to automate the creation and storage of strong passwords. I passed all the account passwords I wanted to create to this function. It leverages OpenSSL for random generation and stores the passwords in Google Secret Manager.
Customizing Ghost Setup Parameters
Since I couldn't rely on an interactive installer custom_ghost_setup_parameters() was designed to assist with a no-prompt Ghost installation. The function starts with the required URL for the Blog and optionally adds Mailgun settings, if provided. It securely syncs this data from the local machine to the cloud via Google Secret Manager. This approach is particularly useful for safeguarding sensitive information like Mailgun usernames and passwords.
Generating SSH Keys and Creating Service Accounts
I needed a function to manage SSH keys and service accounts, which I named create_keys(). This function generates SSH keys, creates a Google Cloud service-account with a secret, and saves essential variables for later use. I set up SSH keys for convenient server access and left the configuration in place since users would likely need to access their instances later on.
To edit config.production.json and perform other tasks on your VM, use your local SSH keys. Just customize the commands below to SSH in to your instance.
Creating the VM
I automated VM instance setup with a startup script that runs on boot. The script initializes variables, sets up accounts and directories, and ensures SSH connectivity. It starts the instance as an E2-Medium running Ubuntu 22.04 with a 30GB persistent disk. The script sets up firewall rules for http/https and outgoing mail, while also pulling secret passwords from Google Secret Manager to establish both the root and service-account passwords. Unfortunately, I found that the I had to disable secure-boot, shielded-vtpm and shielded-integrity-monitoring due to server performance issues on an E2-Micro instance.
Automating Ghost Install with User-Provided Settings
Since the Ghost installation could only run in SSH if I passed it all the setup variables the VM fetched ghost_install_setup_parameters-$INSTANCE_NAME from Google Secrets. It contained custom parameters like --URL and Mailgun settings, and passed them directly to the ghost install command. While I would have preferred to have Ghost install run interactively this streamlined approach handles the MySQL, Nginx, SSL, and systemd setups, effectively making the installation process hassle-free and customized. So long as you like the settings I selected. 🤣
Optimizing SQL on the Server
I've implemented several SQL optimizations to enhance server performance and resolve potential issues. Firstly, I automated the setup of MySQL root credentials and flushed privileges using a temporary SQL commands file. After executing the SQL commands, I reset MySQL to production settings
Another key optimization was disabling MySQL's performance schema to lower its memory usage. I modified the MySQL configuration file to include these settings, making it easier to access MySQL from the command line with root user credentials.
Lastly, I tackled a puzzling MySQL error that could occur post-Ghost setup. To address this, I created another temporary SQL file with commands designed to resolve this specific issue, executed the commands, and then deleted the temporary file.
Managing Non-Sensitive Variables and Downgrading Instances in Google Cloud
Preserving Non-Sensitive Variables Across SSH Sessions
When SSH'ing into a remote server, local variables are lost, requiring a strategy to retain necessary information. I wrote non-sensitive variables like INSTANCE_NAME, ZONE, and REGION to a temporary file, making it easy to read them back into the script and to avoid making more network calls.
Automating Instance Downgrading and Static IP Assignment
Once the VM was setup on the E2-Medium I wanted it to be more cost-effective to run 24/7, I automated the process of downgrading it to a free-tier E2-Micro instance and assigning it a static IP address.
By automating these tasks, the script not only installs the Ghost blog but also optimizes the instance for cost and performance, demonstrating a comprehensive approach to VM management on Google Cloud.
Knowing When to Stop
While the temptation to keep adding features like DNS management and SSL configuration was strong, every project needs a logical endpoint. A part of me wanted to convert this to a generic installer to support other software or even GitHub repo URLs. But, I came to my senses that those enhancements could be future iterations; for now, the focus was to automate as much of the Ghost setup process as possible as part of a Google Cloud learning exercise. Maybe someone else will find this useful and extend it!
Conclusion
This project served as a hands-on deep dive into Google Cloud, and allowed me to offer a free alternative to the typical $6/month Ghost hosting (such as the 1-click installer offered at Digital Ocean). Though my script might not be the most efficient setup method for Ghost, it was a valuable learning experience for me. It also led me to explore writing a Cloud Function for VM monitoring and a Cloud Run Function to restart a VM. And, I built a Ghost software updater inspired by Amazon's rolling deployments. Suffice to say it's been a solid stepping stone to understanding Google Cloud better. Onwards to other projects!