By Dan Moren

December 22, 2021 2:57 PM PT

Mystery machine: Automating puzzle hunting

Since 2005, I’ve partaken in a peculiar pastime every January¹: the MIT Mystery Hunt. This weekend-long competition sees teams attempt to solve around two hundred puzzles involving everything from obfuscated songs to labyrinthine text adventures. It’s an incredibly fun (and incredibly challenging) event, and one that, in addition to critical and lateral thinking skills, also requires a lot of logistics.

Over the last couple years—especially as both last year’s hunt and this year’s upcoming hunt have been impacted by COVID—I’ve taken on a larger role in helping set up the technology for our Mystery Hunt team of about 25 people.

While I’ve mentioned this obliquely in my previous post on Integromat, I thought it might be interesting to run down the various tools we’re using to automate what’s informally called our “puzzle collateral” system. This largely involves metadata about puzzles that help us track our solving status, as well as the collaborative systems we use in the actual solving. In the past few years, those systems have become inextricably linked, especially as puzzling moves to a remote-only footing. So here’s an overview of what we’re currently using to aid our process.

Discord

The 2021 Mystery Hunt was the first to go fully remote.² In the past, some teams have had remote components of various sizes, to help far-flung members who can’t travel still participate, or to take advantage of timezones to provide a 24-hour puzzle-solving crew.

Though our team doesn’t generally have a large remote presence, we’ve still traditionally needed a central communication system to help manage our collaboration and information flow, and also provide access for parts of the hunt where people might be offsite. In past years, we’ve relied on Slack, but with the shift to a fully remote hunt, 2021’s hunt saw us shift to Discord, for two major reasons. First, Discord’s integrated—and, more importantly, free—multi-person video and audio chats make it much easier to have a collaborative environment. Second, integration with automation tools seems to be more accessible and more powerful with Discord.

In addition to a handful of channels for general chatting, non-hunt socializing, burritos³, and administration, our Discord contains a text channel for each puzzle that we unlock, organized into channel categories based on their meta puzzles.⁴

Rather than create a video/audio channel for each puzzle that we unlock, we opted instead to create a half dozen “conference rooms” where ad hoc groups could gather while working on a specific puzzle. That ended up being easier to manage, especially for the size of our group, and there’s always the ability to create another conference room as needed.

Google Sheets

The backbone of puzzle solving is collaborative data crunching. For that, there’s nothing better than a Google Sheet. For each puzzle, we create a spreadsheet that’s put in a Google Drive folder for the related meta puzzle. Multiple users can join those spreadsheets and view the same data as it’s manipulated (and, if necessary, duplicate and experiment with data on their own). Links to the Google spreadsheet for a particular puzzle are included in the topic of the Discord text channel for that puzzle.

We also use a Google Sheet as our “master puzzle list.”⁵ This is a Google Sheet that contains the puzzle name, its associated meta, the URL for the puzzle itself, the Discord channel name and ID, the Google Sheets ID and URL, and the solution (if any). But how to populate that?

Google Forms

A pair of Google Forms allow us to keep our puzzle list easily updated. At any given time, someone on the team is the “puzzle lead”, in charge of inputting new puzzles into our master list as they’re unlocked. They fill out a Google Form with the puzzle name, the associated meta (or nothing, if it’s a new meta), and the puzzle URL and those results are automatically dumped into the master puzzle list. There’s also a puzzle solver form that allows the puzzle lead to input the answer to a puzzle once it’s been solved.

In both of these cases, I rely on a little Google Apps Script magic to create a drop-down menu in the form where the options are themselves pulled from data in a spreadsheet. That helps eliminate data errors that might come from, for example, a typo in a puzzle name. The one downside is that those scripts can only be triggered at maximum once a minute, which means there can be some lag time between inputting a puzzle and having it show up in the drop down list, although it’s rarely a problem that actually has an effect, given the time between these actions.

Integromat

Now for the duct tape that binds the galaxy together. Integromat is a web-based workflow system that lets you have various services communicate with one another. It’s the main workhorse of our collateral system, because it allows us to take information from the master puzzle list, create the necessary ancillary materials, and then update any records accordingly. We use two Integromat scenarios, of which the more complicated is the Puzzlemaker workflow. Here’s a quick look:

To sum up, the Puzzlemaker workflow is triggered by a webhook that’s manually clicked by the current puzzle lead. (Previously we used a system where it watched for new records every five minutes, but that proved to be not only too slow, requiring manual intervention anyway, but also introduced the potential for collisions depending on the current state of the workflow. This year we’re attempting to batch puzzle creation, which also helps save on the number of times the scenario is triggered, and thus any resultant quotas from Integromat.)

When the webhook is triggered, Puzzlemaker looks for new records within the master puzzle Google spreadsheet. It then creates a new Google Spreadsheet for each new puzzle and shares that sheet with our Google Group so that everybody has access. There’s then a series of branches for various actions, not all of which are executed for every puzzle.

If the puzzle is a new meta, a corresponding Discord channel category is created and a record for the meta puzzle and that channel ID are created in our master puzzle list. A Google Drive folder for that meta is also created and placed in a Metas folder.

If the new puzzle is not a meta, the newly created Google Sheet is placed in a Google Drive folder associated with its parent meta.

In every case, the associated meta puzzle for a new puzzle is located within our master list of metas, so that when the next step creates the Discord text channel for the new puzzle, it can be placed under the category associated with that meta (or, if it’s a meta itself, put under a top level “Metas” category). The master puzzle list is then updated with all the new information about Google Sheets and Discord Channels, and a confirmation message is sent to a read-only “puzzle log” channel in Discord.

By comparison, the Solver workflow is much more straightforward.

A webhook is, in this case, automatically triggered by the submission of the relevant Google Form. The puzzle in question is located in the master puzzle list, updated with the answer, and the Discord channel is moved to a category for solved puzzles. Then both our puzzle log channel and out Discord’s general channel are pinged with a celebratory message indicating that a puzzle has been solved, along with the solution.

Grav

The last main tech component for our Mystery Hunt team is a basic website that hosts two main types of information: team-level information that don’t change much during the hunt (such as a team roster, basic tips and reference, and details about the hunt itself), as well as a Puzzle Dashboard.

The latter is an embedded frame that provides a list of our current puzzle status, including whether a puzzle is solved; the puzzle name; links to the puzzle page, Discord channel, and Google Sheet; and the answer, if applicable. This is sort of a central clearing house that can at a glance show what puzzles still need solving, and how we’re progressing on any given meta, as well as provide quick access to puzzle collateral.

To host of all this, we use a basic CMS called Grav. Its flat-file approach avoids the need for a database server, and it’s intended to be lightweight, easily editable, and quick-loading. It also has a robust plugin architecture in the case that it needs to be extended. However, as it doesn’t get a lot of use during the hunt—with the exception of the Dashboard itself—it’s not the most critical piece of infrastructure.

Hunt and ye shall find

And there you have it: one team’s approach to keeping track of puzzles. Our tool usage is tuned for our particular team, of course, and it’s anything but a one-size-fits-all approach. What works for us probably won’t work for most other teams, though we are always on the lookout for new tools and improved workflows that can help us simplify things. But, all of that said, we tend not to tinker too much, especially if something is working pretty well so far. And really, it’s quite astounding what you can achieve with a small amount of off-the-shelf software, most of which is free.

I hope you enjoyed this look into our technology infrastructure. If you’re a fellow puzzler, perhaps it’s of interest to you. And if not, perhaps I at least intrigued you enough to check out the Mystery Hunt for yourself. Happy puzzling!

[Dan Moren is the East Coast Bureau Chief of Six Colors. You can find him on Mastodon at @dmoren@zeppelin.flights or reach him by email at dan@sixcolors.com. His latest novel, the supernatural detective story All Souls Lost, is now available for pre-order.]