It looks like some paid work I was counting on won’t be going ahead, so I’m trying to save a bit of money on cloud hosting. As I previously noted, this resulted in the resurrection of The future of the past, but I’ve also been continuing to slog away at migrating all my old Flask apps and experiments from Heroku to a single Digital Ocean droplet. As of today, I’ve migrated 11 apps. Here’s a few details…

A new (old) home

The first thing I had to figure out was how to group together a series of individual Flask apps so I could easily run and maintain them on a single server, without making major changes to the apps themselves. I decided to go with the application dispatching pattern described in the Flask documentation. This groups the apps within a single Python environment so I had to do some alignment of Python versions and packages, but it wasn’t too hard and having just one virtual environment to manage seems a lot easier in the long run.

The application dispatching pattern configures the server to run one application at the web root ('/'), with the other apps assigned individual sub-paths. This raised the question, what did I want sitting at the root address? Rather than selecting an existing application for the prime slot, I decided to take the opportunity to build a showcase that included details of many of the things I’ve created over the years.

The old Wragge Labs (circa 2012)

I also needed a new domain name. Or did I? Back in the old days, I had a site where I shared many of my tools and experiments – Wragge Labs. In the intervening years, I’d moved or migrated much of the content away and pointed the wraggelabs.com domain to my main site at timsherratt.au. But this seemed like a good opportunity to resurrect it. So if you’d like to have a play around with some of the things I’ve created over the last 30 years, head along to the all new wraggelabs.com!

The new Wragge Labs showcases websites, apps, and experiments from the past 30 years – some useful, some playful, and some creepy…

A surprising amount of the things I’ve built are still working. As I was compiling my list, I made a few running repairs – for example, fixing broken links in Linking history in place and Magic Squares to get them working again. However, some things only exist now in web archives, and others have been broken by the recent actions of the NLA and NAA.

To provide a bit of extra context, I’ve grouped together publications and presentations documenting many of the experiments. These are saved in Zotero, and tagged with the name of the app. When you click on a ‘Related’ link, the details of any linked resources are retrieved using the Zotero API and displayed on a new page. This means I can add new related resources simply by dropping them into Zotero.

Moving house

The process for moving the apps to their new home was pretty straightforward. On my local machine, I copied the code into the new aggregated structure, added any packages needed into a combined requirements file, and created a new top-level app to direct requests. And then I spun everything up and started fixing bugs…

All of the problems were easily resolved. Most involved fixing up paths to static assets, or in navigation links. The only significant changes to the Python code were caused by the deprecation of the .count() method in Pymongo.

To make life a little harder, I decided to take the opportunity to make sure that all the assets – javascript, css, and font files – were loaded from the local system, and not sitting in the cloud. Having everything local should make it easier to maintain the apps in the long term. It was a bit fiddly tracking down where everything was being loaded from, but not too hard.

The only other changes I made were to add some caching to most of the apps, particularly those that make calls to external databases or APIs. I used Flask-Caching with the local file system backend.

To get the new aggregated application working on a Digital Ocean droplet, I followed the instructions on how to serve Flask applications using uWSGI and Nginx. I think the only thing I did differently was to use pyenv to manage Python versions and the virtual environment. To update the app, I use rsync to copy across the code and systemctl to restart it. So far it’s all working pretty smoothly.

Redirecting Heroku

Once the apps were happy in their new home, I needed to redirect the Heroku addresses to wraggelabs.com. It was surprisingly hard to find good documentation on how to do this, so I’ll document my steps in detail in case its of use to others.

There are a few redirect apps for Heroku around, but I decided to use heroku-redirect because it basically just configures and runs Nginx without any additional processing. First I cloned heroku-redirect to my local system, and then for each app I wanted to migrate I followed these steps:

• cd into the heroku-redirect directory
• set the git remote for the app you want to redirect: heroku git:remote -a [app name]
• if you’re not using the lastest Heroku stack, update it: heroku stack:set heroku-24 -a [app name]
• set the url you want to redirect to (without trailing slash): heroku config:set LOCATION=[new url]
• to add the full path to the redirected url: heroku config:set PRESERVE_PATH=true
• I found I had to remove the Python buildpack from the app before updating, I did this using the Heroku dashboard, but no doubt there’s also a CLI command
• I also used the dashboard to add a new nginx buildpack: https://github.com/heroku/heroku-buildpack-nginx.git
• finally I pushed the new app, using --force to replace it completely: git push --force heroku master:main

Of course, you’ll need to have the Heroku CLI installed.

A number of my Heroku apps were using basic dynos (which cost US$7 a month), so once they were redirected, I changed them to used shared eco dynos. Yay – money saved! Hopefully, the redirects won’t push the eco dynos beyond their monthly limit.

More experiments to come?

One of the good things about all of this housekeeping is that it’s got me thinking about new experiments. I used to love Flask and Heroku because it they made it so easy to build and share things. Now I can do the same with wraggelabs.com!