A Common Lisp Web Development Primer, Part 1

Tagged as Linux, LISP, Programming

Written on 2010-09-19 19:57:54

Disclaimer Pt.1: There are many people smarter and more qualified than me when it comes to CL web development. Thankfully, this article is covering basics and my knowledge should be sufficient. Correct me where wrong, of course.
Disclaimer Pt.2: This article will deal more with config files, programming environment setup and the CL web dev landscape with a follow up article to introduce a specific framework, examples, etc.
Edit of Nov 12, 2010: This article has been updated to reflect modern Common Lisp best practices (i.e. Quicklisp).

The Hardware

The first thing to talk about is where our application will be served. This was recently discussed on lisp.reddit. Unlike languages like PHP, Perl, Ruby or Python, shared hosting is not prevalent for lisp though tech.coop is a notable exception. In general, a VPS or similar setup will be required. I've heard good things about people using Slicehost, Linode and ThrustVPS. Personally, I use Linode's smallest instance for $20 a month and have been quite happy with it. I've run hunchentoot, lighttpd, postgres and mysql on it simultaneously without issue but that wasn't under significant load. I'm also aware of at least one startup using Lisp on top of Amazon's EC2. Heck, you may have a server you'd like to run out of your home. For our purposes, I will assume you have a reliable internet-facing Linux box and root access.

The Linux Distro and Programs

Any Linux distribution should be suitable for lisp web server duties. Personally, I would lean towards Archlinux as their default install is quite lean and they keep very recent versions of SBCL(1.0.42), CMUCL(20a) and others packaged. There's even a CCL AUR package (Archwiki AUR article) maintained by Leslie Polzer. Whatever distribution you wind up using to follow along with this series I recommend you also install screen, emacs, sbcl and lighttpd with your package manager. You should also grab the VCS pentafecta of darcs, git, mercurial, subversion and cvs.

Setting up Emacs and SLIME

Note that there are many other, probably better, Emacs+SLIME tutorials out there. Since the original writing of this article, Quicklisp has become the dominant method for acquiring Common Lisp libraries. Instructions for its use are here and the clbuild instructions are maintained below for posterity. First grab quicklisp with curl -O http://beta.quicklisp.org/quicklisp.lisp then load and install it by running sbcl --load quicklisp.lisp, followed by evaluating (quicklisp-quickstart:install), (ql:add-to-init-file) and (ql:quickload "quicklisp-slime-helper"). Finally, add (setq inferior-lisp-program "sbcl") and (load (expand-file-name "~/quicklisp/slime-helper.el")) to your ~/.emacs.

Alternate clbuild instructions
The first thing to do is grab clbuild. At least until quicklisp is released, clbuild will remain the easiest way to get all the necessary lisp libraries to get cranking on web development in linux. I like to keep clbuild in ~/builds but place it where you like. Download it with darcs get http://common-lisp.net/project/clbuild/clbuild. Then cd into the clbuild directory and make it executable by running chmod +x clbuild. I'd also add the directory to your path in .bashrc or add an alias like alias clbuild='/home/redline/builds/clbuild/clbuild'.

With that done, it's time to start grabbing libraries. First, get SLIME by running clbuild update slime. Then you'll want to run clbuild slime-configuration and stick that in your ~/.emacs file, taking care to change the (setq inferior-lisp-program "/home/.../.../clbuild/clbuild lisp") to (setq inferior-lisp-program "sbcl") or "/usr/bin/sbcl" or whatever is appropriate.
End clbuild-specifics

At this point you should be able to ssh into your development server, run emacs -nw (for no-window-system/terminal mode) and then type M-x (alt-x) slime and enter to get to a lisp prompt.

Getting the Lisp Libraries

After talking with Leslie a bit, I'll be using weblocks-dev over weblocks-stable. Weblocks-dev use is encouraged over stable at this time. Quicklisp uses weblocks-dev already and makes this insanely easy, just evaluate (ql:quickload 'weblocks). Done.

clbuild specifics
If you'd like to use weblocks-dev, open /path/to/your/clbuild/wnpp-projects in your favorite text editor and change the following:
1) Find elephant and change it's darcs repo from blah/blah/blah/elephant to blah/blah/blah/elephant-1.0
2) Find weblocks and change it's hg repo from http://www.bitbucket.org/skypher/weblocks-stable/ to http://www.bitbucket.org/S11001001/weblocks-dev/
3) Find cl-prevalence and change it's repo to cl-prevalence get_hg http://www.bitbucket.org/skypher/cl-prevalence/
Then run clbuild update weblocks and, if prompted about whether or not to download dependencies, enter 'y'. Let it work it's most excellent magic.
End clbuild-specifics

The Framework Selection

There are a multitude of ways to do web development in Common Lisp. There are web servers such as Araneida and Portable Allegroserve (both effectively unmaintained), Hunchentoot (which is dominant in the way *Ediware* often is), Apache with mod_lisp, relatively obscure or undocumented combination server/frameworks like Antiweb, Symbolicweb/SW-HTTP and Teepeedee2 and frameworks like Weblocks, UCW and RESTAS.

I wanted something relatively commonly used and well-documented but I wanted a framework as opposed to just using libraries like CL-WHO, Parenscript and Postmodern on top of hunchentoot. Since UCW already has a blog series and I've worked with Leslie on Paktahn a while, Weblocks was a natural choice for me.

The App Setup

I already run a wordpress blog and some other stuff on a lighttpd server on my Linode. Consequently, it made sense to just map a subdomain to my lisp experiments and leave the rest alone. To do this with lighttpd, add the following to /etc/lighttpd/lighttpd.conf:

$HTTP["host"] =~ "testbed.redlinernotes.com" {
              proxy.server = ( "/" => ( ( "host" => "127.0.0.1",
                                          "port" => 4242 ) ) )
}

Now you wouldn't want your webapp to not restart if you had to reboot the server would you? Of course you wouldn't. I've taken a cue from Xach and daemonized it via screen as follows:
Open /etc/rc.local in your favorite text editor and insert a line similar to su redline -c 'screen -d -m -S screenslime -c /home/redline/webapps/screenrc'. This will ensure that the redline user starts a screen instance in detached mode with the name "screenslime" when the system boots that will use /home/redline/webapps/screenrc as it's configuration. If you don't know what any of that means, don't worry. It means screen is cool and you want it.

Now, you should add something like the following to whatever file you listed as your screenrc:

chdir /home/redline/projects

screen emacs -nw
screen sbcl --userinit /home/redline/.sbclrc --load /home/redline/webapps/init.lisp

This will ensure screen defaults to the /home/redline/webapps directory, starts an emacs instance in no-window-systems mode in screen window 0 and starts sbcl loading your .sbclrc and a lisp init script for your webapp(s) in screen window 1.

Next, we need to actually write the init file for your webapp. For now, it will be quite simple as I'm just playing with weblocks. In the next article, we'll build something (probably un) interesting. In your init.lisp file (or whatever you called it) insert something like:

(ql:quickload '(weblocks swank))

(setf swank-loader::*contribs* '(swank-c-p-c swank-arglists
                                 swank-fuzzy swank-fancy-inspector
                                 swank-package-fu))
(swank-loader::loadup)
(swank:create-server :dont-close t
                     :port 4010
                     :coding-system "utf-8-unix")

This will ensure weblocks loads and swank serves on port 4010 so that we can use SLIME to connect and work on the running system. Note that you could've also put (load "/path/to/my/.sbclrc") or inlined the following as the first line(s) in your init file and avoided the --userinit portion of the sbcl invocation in screenrc. My .sbclrc simply points sbcl to the clbuild libraries like so:

(require 'asdf)
(setf asdf:*central-registry* '("/home/redline/clbuild/systems" *default-pathname-defaults*))

Unless you're using clbuild, you won't need this in your .sbclrc but if you are it's important that you do this so that sbcl can find the lisp libraries we downloaded with clbuild. If you don't, it'll be looking in ~/.sbcl/systems.

Finally, I would also add a bash alias to your ~/.bashrc to get you right back to where you were with screen+SLIME. Mine is alias webslime='screen -dR'. I also added stty -ixon to my .bashrc as detailed in my last post because screen was capturing keystrokes I wanted sent to emacs. Xach pointed out that this could be toggled in screen with C-a C-f but I preferred having it as a default.

See, now that was mostly painless, wasn't it? Next time I'll cover the basics of weblocks and develop a simple starter application. Or if I'm feeling particularly lazy, maybe I'll just walk us through the simple-blog example-app. Cheers.

• ← Previous
• Next →