Devlog, January 2025

I have all manner of visually-interesting, arty projects I want to do, but they’re all stuck behind one bigger one that I can’t stop obsessing over even though it’s extremely self-absorbed and will be interesting to nearly no one. Basically an archive and timeline of my life, similar to Buster’s Life in Weeks, but more data-driven and detailed.

(Another reason I want to do this first is that all the arty projects are much more difficult.)

Getting this together will require a detailed and time-consuming slog through my digital and physical detritus, which sounds like no fun at all, so I’ll put that off and focus on the parts I can automate instead. I’ve got location data in Swarm going back to 2009 (sadly no further), and flight data from Tripit going back to 2008. Let’s combine those into “trips” which can be the first category of events in this life archive.

I spent a lot of January on this and it took many twists and turns.

• write a TypeScript/Deno script that pulls down the Swarm check-ins and stores them in SQLite
• just for fun, reverse engineer the API on the Michelin Guide site to check which restaurants I’ve been to have Michelin awards, discover that they have no rate limit on the API which is great when you’re searching for thousands of things
• while you’re at it, do the same for hotels in the Michelin Guide
• realize that writing SQL is no fun at all, remember that ORMs are a thing, discover Prisma
• well if you’re going to do all that you may as well use Postgres
• write another script that imports a CSV from Flighty into a table
• realize that all the dates are stored as times local to each airport, so have fun converting all those to UTC
• realize that a whole bunch of flights are missing, manually copy them over from TripIt to Flighty
• write another script to figure out what a “trip” is, using some complex clustering logic that assumes big clusters of Swarm check-ins are “home” and outliers are trips, this mostly works after several iterations
• add the flights to the trips, which could simply be flights that overlap the time range of the check-ins, but that would be too simple, instead link each flight to a check-in at an airport, taking canceled flights into account as well
• think about how to structure the project, the key point being you want the same Prisma schema file accessible to the front-end and the data scripts
• try two repos with a manual sync, a single monorepo, and finally the same Nuxt app repo you started with, just with a folder of scripts in it
• consider how to get the data into the front end, start with a custom content source (feels weird to query the db on build of the site), try querying the db straight from the front-end with Prisma (doesn’t work that way), write a Nuxt API that does the database queries there (too much code for a simple thing), discover TRPC to write the API instead, which feels truly magical
• finally write a basic web page that displays a trip, combining the scraped data from the database with artisanal markdown that describes it

Yeah, it was a whole journey. Next post will include a not-yet-very-impressive demo.