Advent Reflections

It's been a busy start to 2022. I'm working as an Engineering Manager for the first time and enjoying it but it's been easy for other things to slip through the cracks. For example, I told myself I would write a post on Advent of Code several weeks ago. So I'm sitting down to write about it now before I forget any more details.

Background

This is the second time I've attempted Advent of Code. The first time was in 2020 and I enjoyed it a lot but ran out of gas around day 10. I was pretty distracted with a Flamingo Squad project I can't recall and probably a bit burned out. Both years I've written my solutions in Common Lisp.

Advent is interesting. I get enjoyment from different things on different days. Some problems I just enjoy seeing how much I can optimize Common Lisp, or writing solutions in a few different styles if the problem is simple and seeing the differences in how they are compiled and allocate memory. Other problems I'm much more satisfied by trying to see how "pretty" a solution I can write, either using constraint solving tools like Screamer or a pipeline using threading macros and so on.

I enjoy the social aspect of AoC and having a leaderboard with some mutual friends and coworkers. It's nice to chat about something besides production code with other talented programmers. That said, I have to be pretty careful to avoid judging myself. I have to consciously remind myself that my goal isn't to "win the race" and not worry too much if I struggle to solve a problem elegantly.

Advent 2021

There were two things I wanted to try and do differently this year from last year. The first was just to go as far as I could and not worry about racing. The second was to experiment with literate programming tools and try to do a better job documenting my work.

I think the results were a mixed bag. I got through day 11 so I powered out at around the same point. I mostly worried less about the race but I still cared a lot about finishing each problem on the day it became available and definitely got discouraged once or twice when I didn't like my approach. On the other hand, I had a good time and learned a few things so it's a good investment overall.

Here's the current state of the generated site. You can see I didn't wind up embedding the source for the different functions so it can't properly be called literate but the [function] label (or equivalent) next to all the exported symbols can be clicked to jump to the source on github.

Takeaways

MGL-PAX

I have been meaning to play with mgl-pax for a long time. Like ... probably several years? There are blog posts about it as far back as 2014 and it's been on my radar a long time but I just never seemed to make time for it. Advent seemed like a good opportunity to dive in.

I like the idea of an environment where prose and code are intermingled, so I have a natural attraction to literate programming. This shouldn't surprise you if you've been here before. It also seems important to me that such an environment for authoring programs should be rooted in the development tools and support the prose as a secondary feature (like MGL-PAX) rather than rooted in the prose and supporting the code as a secondary feature (like org-babel). I.e. tangling one or more files to produce my program seems like the wrong way to go to me.

In terms of Advent of Code problems, I'd ideally be able to do the following:

• Keep the prose and code for a given day/problem in a single file
• Easily export the entire project with all days to an easily navigated, well designed web page
• Make it easy to show different versions of the code as well as disassembly or evaluation examples

Editor's Note: The issues I bring up below were resolved before I could ever make a PR or ask the author about them. It seems making PAX more flexible about transcription was in the plans all along.

I think MGL-PAX excels on the first two points and struggles more on the third. It has a feature called transcripts which could plausibly support it but they are an awkward fit. Transcripts allow for including examples that are evaluated when the documentation is generated but there are two issues I have with it:

1. The results are actually embedded in the source code. The argument for this is that they are parsed and checked to ensure the code and docs don't get out of date. I'm interested in embedding details like disassembly though and things like the memory address will change from run to run. The output could also easily dwarf the code for the function itself so shouldn't be directly embedded. (PAX now supports not checking consistency, allowing me to simply dump the output of a form.)
2. The second (much more minor) issue is that there isn't a straightforward way to ask the source definition of a function to be embedded in the doc. MGL-PAX assumes that symbols listed in the doc should be an exported part of the public API and links directly to the source on github at the relevant commit. It's a neat bit of hackery but makes more sense in the case of medium-to-large projects rather than Advent of Code exercises. (Similarly, PAX now has a way of adding "includes" via transcript.)

My only remaining concern is that the navigation and transcription functionality is tied to slime and swank. Hopefully I'll have an opportunity to try them with sly soon and can dig in or report it if there are issues.

In the advent project, the site-building and deployment was trivial. Hacking together a way to generate an overview of all solved problems with performance measurements involved some fiddling but I'm happy with the results.

Evolving my style

For a long time, I've been writing small projects in common lisp, writing a handful of tests, and relied only sparingly on libraries. A little alexandria here, a little cl-ppcre there. There's a place for that but I'm ready to try and cobble together the utilities and extensions to the language that I'm comfortable with. For now, that's alexandria, serapeum, iterate, mgl-pax, and try. Clingon, trivia, screamer, and fset wait in the wings for the right problem.

There are plenty of talented lispers around. Two people whose code I've enjoyed reading during advent are death and Steve Losh (aka sjl). They feel like opposites to me if only because sjl has a project dedicated to advent and an assortment of dependencies, macros, and utilities to make advent hacking pleasant. Death by contrast almost always just relies on the language standard and throws his code unceremoniously in a gist. His solutions are often faster than mine but don't sacrifice elegance.

Conclusion

It all boils down to this: I still like lisp, I miss hacking it, and I should read and write more code. My algorithms chops aren't as good as I'd like and I have to make an effort to not get discouraged by my limitations. All the more reason to keep doing advent, even out of season, and learn a few things.