A Common Lisp Web Development Primer, Part 3

posted on 2010-11-22 04:24:03

Disclaimer: What? You haven't already read the first two parts? Feel free to go ahead and do that. The same disclaimers apply. (require 'cl-wdp-part1 'cl-wdp-part2)

Today's Topics

The main topics we'll be covering are Weblocks widgets, forms and views along with a brief example of creating a presentation to use jQueryUI's Datepicker. There will also be a brief aside on what to do after an emergency reboot or power outage relating to cl-prevalence and trivial-timers. By the end the clockwork site will be fully functional.

Form-widgets, Widgets and Views

The User Guide has pretty nice expository summaries of Widgets and Views. The bottom line is that Widgets are what Weblocks pages are composed of and views are different ways to render those widgets. A macro called defwidget is used to construct widgets which are just new classes which inherit from the widget-metaclass. Don't worry if you don't know what that means. Essentially, you define widgets for the data you care about working with and then you define views to render that data, whether as a form for editing, a table for comparing multiple items or some other representation.

Leslie has been working on some new form-widget code intended to remove some of the sharp edges of the current system. As this site is basically just a form it made sense for me to try to use this new code (and Leslie wanted me to help him bang on it). Consequently, one thing you'll need to do is add (load "/path/to/weblocks/contrib/lpolzer/form-widget.lisp") to your init.lisp file that runs on startup. It should be placed after weblocks is loaded and before the loading of clockwork.

Last time we defined a reminder class with slots for the data we really care about: a list of emails, a title and summary and timestamps for when to send that message and when the event itself occurs. However, it won't do to ask our users to input timestamps or to tell us the "email" for their cell phone's SMS gateway. To that end, we'll define a view to gather the information we really need to construct the timestamps and other parts of the reminder. We need to know whether they want to be notified by email, text message or both. We'll need the corresponding contact info, including their cell carrier if they want to be notified by text message. We'll also need to note the event date, timezone and time as well as how long before the event they'd like to be reminded and a message and title. I've also thrown in a "honeypot" field. In theory, spam bots will indiscriminately fill it so we won't get any bogus submissions because the form won't validate. Maybe later we'll replace this with reCaptchas.

So let's get to it and define a view for our form in a new file src/forms.lisp. Insert the following code:

(in-package :clockwork)

(defview reminder-form-view (:type form :caption "Schedule an Event Reminder..."
			     :buttons '((:submit . "Submit")) :persistp nil)
  (send-as :present-as (dropdown :choices '(("An email and a text." . :both)
					    ("Just an e-mail." . :email)
					    ("Just a text." . :text))
				 :welcome-name "How to send it")
	   :requiredp t)
  (email :satisfies 'valid-email)
  (cell-number :satisfies 'valid-cell-number)
  (cell-carrier :present-as (dropdown :choices *sms-gateways*))
  (event-date :present-as (calendar) :requiredp t)
  (event-hour :present-as (dropdown :choices *hour-choices*)
	      :requiredp t)
  (event-minute :present-as (dropdown :choices   '(("00" . 0)
						   ("15" . 15)
						   ("30" . 30)
						   ("45" . 45)))
		:requiredp t)
  (timezone :present-as (dropdown :choices *timezones*)
	    :requiredp t)
  (remind-me :present-as (dropdown :choices '(("At the event" . 0)
					      ("5 minutes before" . 300)
					      ("10 minutes before" . 600)
					      ("15 minutes before" . 900)
					      ("30 minutes before" . 1800)
					      ("45 minutes before" . 2700)
					      ("1 hour before" . 3600)
					      ("2 hours before" . 7200)
					      ("1 day before" . 86400)
					      ("2 days before" . 172800)
					      ("1 week before" . 604800)
					      ("2 weeks before" . 1209600)))
	     :requiredp t)
  (subject :requiredp t)
  (summary :present-as (textarea :rows 5))
  (honeypot :label "Leave this blank" :satisfies #'null))

(defparameter *timezones*
  '(("UTC-12:00 (Eniwetok, Kwajalein)" . -43200)
    ("UTC-11:00 (Midway Island, Samoa)" . -39600)
    ("UTC-10:00 (Hawaii)" . -36000)
    ("UTC-09:00 (Alaska)" . -32400)
    ("UTC-08:00 (Pacific Time)" . -28800)
    ("UTC-07:00 (Mountain Time)" . -25200)
    ("UTC-06:00 (Central Time)" . -21600)
    ("UTC-05:00 (Eastern Time)" . -18000)
    ("UTC-04:00 (Atlantic Time, Caracas)" . -14400)
    ("UTC-03:30 (Newfoundland)" . -12600)
    ("UTC-03:00 (Brazil, Buenos Aires, Georgetown)" . -10800)
    ("UTC-02:00 (Mid-Atlantic)" . -7200)
    ("UTC-01:00 (Azores, Cape Verde Islands)" . -3600)
    ("UTC+00:00 (Lisbon, London, Casablanca)" . 0)
    ("UTC+01:00 (Berlin, Brussels, Copenhagen, Madrid, Paris)" . 3600)
    ("UTC+02:00 (Kaliningrad, South Africa)" . 7200)
    ("UTC+03:00 (Baghdad, Moscow, Riyadh, St. Petersburg)" . 10800)
    ("UTC+03:30 (Tehran)" . 12600)
    ("UTC+04:00 (Abu Dhabi, Baku, Muscat, Tbilisi)" . 14400)
    ("UTC+04:30 (Kabul)" . 16200)
    ("UTC+05:00 (Ekaterinburg, Islamabad, Karachi, Tashkent)" . 18000)
    ("UTC+05:30 (Bombay, Calcutta, Madras, New Delhi)" . 19800)
    ("UTC+05:45 (Kathmandu)" . 20700)
    ("UTC+06:00 (Almaty, Colombo, Dhaka)" . 21600)
    ("UTC+07:00 (Bangkok, Hanoi, Jakarta)" . 25200)
    ("UTC+08:00 (Beijing, Hong Kong, Perth, Singapore)" . 28800)
    ("UTC+09:00 (Osaka, Seoul, Sapporo, Tokyo, Yakutsk)" . 32400)
    ("UTC+09:30 (Adelaide, Darwin)" . 34200)
    ("UTC+10:00 (Eastern Australia, Guam, Vladivostok)" . 36000)
    ("UTC+11:00 (Magadan, New Caledonia, Solomon Islands)" . 39600)
    ("UTC+12:00 (Auckland, Fiji, Kamchatka, Wellington)". 43200)))

(defparameter *hour-choices*
  (loop for i from 0 to 23
     collecting `(,(format nil "~d" i) . ,i)))

So here we're defining a view called reminder-form-view and passing in a list of arguments *about* the view as well as a list of fields in the view. In the list of arguments about the view we note that it's a form and we don't want to persist the form contents directly. We use a variety of keywords in the list of form fields to get the behavior we want including :present-as, :requiredp, :satisfies and :label. Present-as allows us to make something a dropdown or any other defined presentation. Note that some presentations do take arguments. Dropdown in particular takes a list of dotted pairs representing the Dropdown choice and it's corresponding value. Requiredp does what you'd expect and marks a form field as required. Satisfies takes a lambda or the name of a function which will validate the field's data. By default, the view will "humanize" the field names and use those humanized names as labels. If you want a different label for some reason, you can achieve that with the :label keyword.

Now we have a form that takes all the data we need to construct a reminder but we still need to validate the emails and cell phone numbers. Additionally, we'll need to write helper functions to construct the email list and timestamps that the reminder's emails and timestamp slots will be set to. Consequently, add this code to the bottom of the file:

(defun valid-email (user-input)
  "Ensure that there is an @ and a . and input not containing @s before and after each."
  (or (cl-ppcre:scan "^[^@]+@[^@]+\\.[^@]+$" user-input)
      (values nil "Your email must have an @, a . and text before and after both.")))

(defun valid-cell-number (user-input)
  "Ensure that only numbers are given and there are at least 10."
  (or (cl-ppcre:scan "^[0-9]{10,}$" user-input)
      (values nil "Your number must have only numbers and at least 10 of them.")))

(defun get-emails (form-data)
  (with-form-values (send-as email cell-number cell-carrier) form-data
    (let ((sms-mail (concatenate 'string cell-number "@" cell-carrier)))
      ;; this was an ecase with keywords but weblocks converts
      ;; the keywords to strings somewhere in form submission
      (cond ((string= send-as "BOTH") (list email sms-mail))
	    ((string= send-as "EMAIL") (list email))
	    ((string= send-as "TEXT") (list sms-mail))))))

(defun get-timestamps (form-data)
  (with-form-values (event-date event-hour event-minute
		     remind-me timezone) form-data
    (let* ((hour (parse-integer event-hour))
	   (minute (parse-integer event-minute))
	   (reminder-time-period (parse-integer remind-me))
	   (timezone (parse-integer timezone))
	   (datestring (split-sequence #\- event-date))
	   (day (parse-integer (first datestring)))
	   (month (parse-integer (second datestring)))
	   (year (parse-integer (third datestring)))
	   (event-time (encode-timestamp 0 0 minute hour day month year :offset timezone)))
      (list event-time
	    (timestamp- event-time reminder-time-period :sec)))))

The validation functions are ORs with the function testing the input as the first clause and a VALUES form returning nil (a failed submission) and an appropriate error message as the second. The helper functions use the with-form-values macro to grab the relevant fields of the form and construct the resulting slot. Get-timestamps is rather nasty but we're essentially just grabbing all those fields pertaining to the time, parsing the integers from them and passing those on to the appropriate timestamp functions in the local-time library.

A Calendar Presentation

It would certainly be better to have a nice calendar than have users enter dates as strings and then try to validate them and God forbid we roll our own given the number of Javascript calendars already out there. Since it's fairly well established I opted for the jQueryUI Datepicker. Previously to use Javascript libraries you needed to download them and place them in your Weblocks project's pub/script folder but thanks to a quick patch by Leslie Polzer remote dependencies are now also supported. In case you didn't read the previous article, when you first start a project with (wop:make-app 'name "/path/to/app") weblocks generates a defwebapp form and a basic package for that app along with setting up a store and some basic resources. To include the jQuery code on our page we'll modify our defwebapp form in clockwork.lisp like so:

(defwebapp clockwork
    :prefix "/"
    :description "Fire-and-Forget Event Reminders"
    :init-user-session 'clockwork::init-user-session
    :autostart nil                   ;; have to start the app manually
    :ignore-default-dependencies nil ;; accept the defaults
    :hostnames '("clockwork.redlinernotes.com")
    :dependencies '((:stylesheet "http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.5/themes/ui-darkness/jquery-ui.css")
		    (:script "http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js")
		    (:script "http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.5/jquery-ui.min.js")
		    (:javascript-code "var $jquery = jQuery.noConflict();"))
    :debug t)

First let's note that this will include these dependencies in the "HEAD" of every page. If you want to have per-widget dependencies Weblocks does support that by (if I'm not mistaken) defining a widget-public-dependencies method specialized on that widget. Since we only have one page in this app anyway we'll just list them here. We've done that by adding entries to the dependencies list that use Google's CDN to supply the minified jQuery and jQueryUI libraries along with a stylesheet for the jQueryUI stuff. We added the :hostnames argument which specifies that requests to any host besides those listed are to be ignored. This is particularly helpful if you're running multiple webapps off of one server but want them to share a Lisp image and port rather than fire up separate Hunchentoot servers for each one. Additionally, we're inlining the js code that calls jQuery.noConflict() in the header because Weblocks uses prototype and scriptaculous out of the box and jQuery will happily steal the $ global variable from Prototype which causes all sorts of havoc. While there is interest in removing the prototype and scriptaculous dependencies it hasn't happened yet. It would be greatly appreciated if any developer felt like taking a little time to tackle this.

So now that we've included the JS code, let's write a presentation. The presentation lets us add the code :present-as (calendar) to a slot in our View and have it render as a calendar. This presentation will be a little different as we're using Leslie's new form-widget code. A more traditional coverage of presentations can be found in this blog post. Create a new file in the src directory called calendar.lisp and insert the following code:

(in-package :clockwork)

;; calendar presentation
(defclass calendar-presentation (input-presentation)
  ())

;; calendar form-widget code
(define-widget calendar-field-widget (field-widget)
  ()
  (:default-initargs :parser (lambda (raw-value)
			       (values t raw-value))))

(defmethod field-presentation->field-widget-class ((presentation calendar-presentation))
  'calendar-field-widget)

(defmethod render-field-contents ((form form-widget) (field calendar-field-widget))
  (with-html
    (:input :type "hidden" :name (name-of field) :value (datestring))
    (:div :id "datepicker"
      (send-script '($jquery (lambda ()
			       (ps:chain ($jquery "#datepicker")
					 (datepicker (ps:create date-format "dd-mm-yy"
                                                                min-date 0
								on-select (lambda (date inst)
									    (ps:chain ($jquery "[name=event-date]")
										      (val date))))))))))))

(defun datestring ()
  (subseq (format-timestring nil (now) :format '((:day 2) "-" (:month 2) "-" :year)) 0 10))

So what's going on here? First we define a calendar-presentation class and a calendar-field-widget class along with a simple method to map the presentation onto the widget-class. Then we write the bulk of the code, a render-field-contents method which generates the HTML and Javascript. We'll use a hidden input field with a name equal to the field in the view that gets initialized to today's date. That will be followed by a div containing the Javascript code for the datepicker written with Parenscript (indentation suggestions welcome) which sets the hidden input field whenever a date is selected.

Putting it all together...

Now that we have all the pieces we need we can hook them together. You'll note that a src/init-session.lisp file already exists and contains a defun for init-user-session. This function sets up the widget tree and creates a new session when users visit the site. Remove the old definition and insert the following into the file:

(defun init-user-session (root)
  (setf (widget-children root)
	(make-reminder-form)))

(defun make-reminder-form ()
  (let ((reminder-form (make-instance 'form-widget :on-success 'submit-reminder-form)))
    (form-widget-initialize-from-view reminder-form 'reminder-form-view)
    reminder-form))

We're just defining a separate function (make-reminder-form) to create our form instance here rather than defining it inline in the init-user-session code. Make-reminder-form itself creates an instance of the form-widget class which runs a function called submit-reminder-form when the form is successfully submitted (i.e. passes validation, etc). Note that we have not yet defined submit-reminder-form. Because the form is really based on a view and not a widget or class we want to persist we'll use form-widget-initialize-from-view in conjunction with the reminder-form-view we defined earlier. Note that you may need to restart the webapp after redefining the init-user-session function. Run (restart-webapp 'clockwork) and check the homepage. You should now have a nice form complete with jQuery Datepicker. But of course, nothing useful happens on submission. Time to fix that by going ahead and defining submit-reminder-form. Open src/init-session.lisp back up and insert the following:

(defun submit-reminder-form (widget)
  (let ((new-reminder (create-reminder widget)))
    (schedule new-reminder)
    (persist-object *clockwork-store* new-reminder))
  (reset-form-widget widget))

(defun create-reminder (form-data)
  (with-form-values (subject summary) form-data
    (let ((timestamps (get-timestamps form-data)))
      (make-instance 'reminder
		     :emails (get-emails form-data)
		     :title subject
		     :summary summary
		     :timestamp (first timestamps)
		     :at (second timestamps)))))

Note that I wrote this in a way that's fairly natural to Lisp. I wrote submit-reminder-form in what almost resembles pseudocode and ensured it expressed my intent before worrying about writing a helper function to make that possible. So submit-reminder-form creates a new-reminder by passing the widget to create-reminder, schedules and saves that new-reminder in the Prevalence store and then resets the form. To make this possible, create-reminder uses with-form-values to scrape out the subject and summary from the form, then we grab the timestamps and emails using the functions we wrote for that earlier and instantiate the reminder object accordingly. At last the site is fully functional for sending email or text message reminders!

Recovering from Failure

We haven't covered what to do in case of an emergency reboot or other failure. Since everything is persisted by prevalence on submission the *clockwork-store* will still have our reminders. All we have to worry about is ensuring that all the timers get rescheduled. This is so simple it hurts. Reopen src/reminder.lisp and add the following code to the bottom of the file:

(defun recover-reminders ()
  "A function to reschedule reminders after a reboot. Based on testing,
any that expired during the reboot will be sent when the schedule method is called.
Better late than never, right?"
  (mapcar #'schedule (find-persistent-objects *clockwork-store* 'reminder)))

Calling (recover-reminders) will schedule all the reminders in the store and whether Linux, SBCL or trivial-timers is to thank, a timer that's scheduled in the past will trigger immediately so you don't have to worry about some being lost during the reboot itself. Just add #:recover-reminders to the export list in the clockwork defpackage in clockwork.lisp and then call it after you load clockwork in your init.lisp file that runs when the server starts. Here's my init.lisp as an example.

Next time...

At this point we've seen a tiny bit of the Store API and the default Prevalence store and learned a little about widgets and views. We still need to cover actions and navigation/dispatchers and it wouldn't hurt to demonstrate user registration and authentication as well as use of a SQL backend. Sooner or later we'll get around to all of those things.

Right now I'm working on a postmodern backend for weblocks and may also work on styling, polish, error handling and potentially a new feature or two for clockwork. Based on what gets done, the next article will either cover the beginning of a new project or continued improvements to clockwork.

A Common Lisp Web Development Primer, Part 2

posted on 2010-11-16 03:19:24

Disclaimer: This blog will not teach you Common Lisp. That is a (mostly) solved problem. See Peter Seibel's Practical Common Lisp and Peter Norvig's PAIP. Other oft-recommended texts include Keene's OOP in CLOS, PG's On Lisp and Kiczales et al's The Art of the Metaobject Protocol. This blog will also not teach you good style, I'm too young for that. It hopefully demonstrates non-atrocious style though. I'm learning web development as I go, so don't count on expert understanding there either.
Disclaimer Pt. 2: For the foreseeable future all these projects will be weblocks-based. If that's not your cup of tea you can check out the RESTAS docs and examples, Felideon's blog on UCW, Adam Petersen's slightly bitrotted sans framework article or "defect" to Clojure and look at all the Compojure stuff and maybe Sandbar.

Introduction

It's taken far longer than I hoped to get this second article off the ground. For those of you who missed Part 1, look here and if you'd rather see code than this article's commentary, the code is available on github. It's worth noting that Part 1 was originally written with clbuild in mind but has since been updated with instructions for quicklisp also. Part 2 details the construction of Clockwork, a simple clone of the now defunct yourli.st, an email reminder service. Clockwork allows you to schedule a reminder and brief note which is sent to you by email or text message at the predetermined time. Right now international numbers aren't supported but I'd be happy to see patches.

Future Plans

There's still a good amount of stuff in the TODO and I have further projects in mind after this. Part of the reason this article was so long in coming is that I've been helping Leslie by testing out his new form-widget library. The other reasons are that I have school and (until recently) was working plus I'm learning web development as I go. The next article will go into polishing this application and will likely be much more Javascript and CSS than Lisp. I'm working on getting a postmodern backend written, tested and merged into Weblocks right now. Once that's done I plan to continue this series by developing a RESTful blog that can import entries from Wordpress and maybe crosspost to livejournal as that should prove more interesting...but like Linus' said, "Talk is cheap, show me the code".

Resources, Libraries and Project Skeleton

Weblocks docs are not in an ideal state and hopefully this blog series will help that some. Four things worth being aware of for a beginner are the Google group, the TINAA-generated docs, the User Manual and User Guide. As usual, googling specific concepts will lead you to blog entries and mailing list posts that may or may not be out of date.

We'll begin by using weblocks helper to create a project skeleton by evaluating (wop:make-app 'clockwork "/home/redline/projects/"). If you've been following along since Part 1, you'll also want to push that path onto the ASDF central registry so you can use quicklisp to load the clockwork system in your server init file (~/webapps/init.lisp). (push "/home/redline/projects/clockwork/" asdf:*central-registry*) will do the trick. Then add a (ql:quickload '(clockwork)) line at the bottom of the file followed by (clockwork:start-clockwork :port 4242). At this point, you should be able to reboot the server and run screen -dRR or similar to get a screen session with emacs and an sbcl instance with clockwork and swank running. They'll be in different windows which you can switch to with C-a (control-a) and the window number. Numbers start at 0. Enough of that, this isn't a GNU Screen tutorial. Go to the emacs instance and run M-x slime-connect making sure to change the port to that specified in the init file. At this point, you're connected to SLIME and can evaluate (in-package :clockwork) and finally get hacking! You should also be able to reach clockwork in the browser at localhost:4242 but there's not much there yet...

To begin with, you'll need some libraries to send emails and schedule reminders to be sent in the future. SBCL provides a Timers facility which we could use for this but it's usually worth doing a little extra work to write portable Common Lisp. To this end we'll use the trivial-timers library as a wrapper and cl-smtp for emails. We'd also like to handle timezones and time arithmetic properly so we'll use the local-time library for that. We'll also be doing some minor string handling which split-sequence winds up being an easy solution for so grab that too. Add those to the :depends-on clause in clockwork.asd in the project directory and then run (ql:quickload 'clockwork) at the REPL. Quicklisp will download and load the new libraries for you. It's that easy. Finally, add :local-time to the :use clause of the defpackage in clockwork.lisp and import the split-sequence symbol from the split-sequence package.

Data and Weblocks Stores

By default Weblocks defines a cl-prevalence backend ("store") which persists data to the "data/" directory in the clockwork project folder. The store itself is defined in conf/stores.lisp and that's where you would go to define additional stores if you wanted them. Weblocks has a special variable, *default-store*, and DEFSTORE sets that variable after defining a store so the last store defined in stores.lisp will act as the default. Weblocks also supports elephant and clsql but for now, we'll focus on other aspects of the framework and delve more into the Store API in a later article. If you're curious now though the Store API is clearly defined and documented.

The only data we really care about is the reminders our users will generate. For our purposes, a reminder consists of some number of recipients, a title, a summary of the event it's reminding you of, the time of the event and how far before the event you'd like to be reminded. A class definition falls out of this rather naturally and we'll add an id slot so prevalence will know how to store it. Create a src/reminder.lisp file, insert the following and add the file to the :components clause of clockwork.asd.

(in-package :clockwork)

(defclass reminder ()
  ((id :reader reminder-id) ;; all classes to be persisted with cl-prevalence need an id slot
   (emails :reader reminder-emails
           :initarg :emails
           :type list)
   (title :reader reminder-title
          :initarg :title
          :type string)
   (timestamp :reader reminder-timestamp
              :initarg :timestamp
              :type timestamp)
   (summary :reader reminder-summary
            :initarg :summary
            :type string)
   (at :reader reminder-at
       :initarg :at
       :type timestamp)))

Now that we have a rough idea of what data we care about, lets look at how to send messages.

Emails and Text Messaging

Text messaging is actually quite simple to support thanks to the SMS gateways run by the major carriers. SMS gateways allow us to send an email to an address which represents a phone number. This is then converted to a text message and forwarded on to the recipient's cell phone free of charge. The downside to this is that it's carrier specific so you have to know the cell carrier of the recipient. It would be nicer to just take a number and figure out what carrier services it but Local Number Portability, among other things, makes this tricky. Whitepages.com has an API for looking this up but their information was out of date for my cell phone and they had a 200 request per API key per day limit. Twilio and Data24-7 offer for-pay APIs but for this example app I'll be staying free and cheap. I'll be coldly forcing my users to select their carrier from a dropdown if they want SMS support.

Since we don't know whether our users really care about their privacy or what kind of data they'll be putting in these reminders, we'll do the responsible thing and send the emails via Encrypted SMTP. I'll be using a gmail account I registered for the service since Google provides free, encrypted SMTP on all their accounts. Let's write a quick helper macro for using it. Create a src/messaging.lisp file, insert the following and add it to the :components clause of clockwork.asd.

(in-package :clockwork)

(defparameter *mail-server* "smtp.gmail.com")

(defmacro with-encrypted-smtp ((&key to subject style
                               (from "cl.ockwork.webdev@gmail.com"))
                               &body body)
  `(cl-smtp:send-email ,*mail-server* ,from ,to ,subject
                       (if (eql ,style :html)
                           (with-html ,@body) ;; TODO: make a nicer render style
                           ,@body)
                       ;; it's worth noting send-email takes a :cc argument
                       :ssl :tls
                       :authentication '(,*smtp-user* ,*smtp-pass*)
                       ,@(when (eql style :html)
                               '(:extra-headers
                                 '(("Content-type"
                                    "text/html; charset=\"iso-8859-1\""))))))

Note that we haven't defined *smtp-user* or *smtp-pass* yet. There are two questions you should be asking. Why a macro and what is it doing? The why is debatable in this case. I wanted the syntax to jump out at me and read a certain way when I use the encrypted SMTP. That's all. The what is fairly straightforward. The macro is syntactically similar to with-open-file and others. It takes keyword arguments for the recipient, sender (with a default value), subject and style of the message along with a message as the body and then sends an email via encrypted SMTP (and cl-smtp's send-email function) with the credentials provided. If the style is :html, it goes to the trouble of specifying a few additional headers.

Since we haven't defined the user and pass, let's do that now. Create a conf/config.lisp file, insert the following and add it to your clockwork.asd.

(in-package :clockwork)

(defparameter *smtp-user* "yourusername@gmail.com")
(defparameter *smtp-pass* "yourpassword")

Obviously, you don't want this puppy in source control. Consequently, I committed it before I filled in the user and pass values then ran git update-index --assume-unchanged conf/config.lisp which tells git to ignore all future changes to the file. Be forewarned, that command might be reversible but I don't know how. Go ahead and add in your username and password, reload the system at the REPL with (ql:quickload 'clockwork) and test it out. You should be able to send yourself an email. Now let's add some helpers for SMS. Return to the src/messaging.lisp file and we'll add a variable defining a mapping of Carriers to SMS Gateway servers and a function for determining if an email address belongs to one of the listed SMS gateways. Add the following code to the bottom of the file.

(defparameter *sms-gateways*
  ;; list is derived from http://en.wikipedia.org/wiki/List_of_SMS_gateways
  '(("AT&T/Cingular" . "txt.att.net")
    ("Alltel" . "text.wireless.alltel.com")
    ("Boost Mobile" . "myboostmobile.com")
    ("Cincinatti Wireless" . "gocbw.com")
    ("MetroPCS" . "mymetropcs.com")
    ("Sprint/PCS" . "messaging.sprintpcs.com")
    ("Sprint/Nextel" ."page.nextel.com")
    ("T-Mobile" . "tmomail.net")
    ("US Cellular" . "email.uscc.net")
    ("Verizon" . "vtext.com")
    ("Virgin Mobile" . "vmobl.com")))

(defun sms-mail-p (email)
  (let ((domain (second (split-sequence #\@ email))))
    (member domain *sms-gateways* :key #'cdr :test #'equal)))

A Few Reminder Methods

We still need methods to send a reminder, delete it when we're through with it and schedule it to be sent at a later time. Let's create those now. Since users aren't required to register to send a reminder, we'll put off letting them delete reminders from the system for now. When they submit the reminder form, a reminder will be instantiated, persisted and scheduled for later sending and deletion. We'd like to offer the user the ability to send reminders by email, text message or both so we'll assume that a list of emails are stored in the emails slot of the reminder and loop through each one, sending it with the macro we defined earlier. Then we'll use Weblocks Store API to delete the object from the datastore. Add the following code to the end of src/reminder.lisp.

(defgeneric send-and-delete (reminder)
  (:documentation "Send the user their reminder as requested and then remove it from the datastore."))

(defmethod send-and-delete ((reminder reminder))
  (loop for email in (reminder-emails reminder) do
    (with-encrypted-smtp (:to email :subject (reminder-title reminder)
                              :style (if (sms-mail-p email)
                                         :plain
                                         :html))
      (reminder-summary reminder)))
  (delete-persistent-object-by-id *default-store* 'reminder (reminder-id reminder)))

Finally, we'd like to schedule the reminder to be sent at a later date. Assuming that the timezone differences are handled beforehand and that the reminder's at slot contains a timestamp for when the message should be sent according to the server's timezone, the local-time and trivial-timer libraries make defining a schedule method pretty easy. We'll just use the local-time library and a let to compute the seconds from the present until the time to send the reminder, make a timer whose function calls send-and-delete on the reminder and schedule it with a delay of the number of seconds computed. The only tricky bit is to pass :thread t to make-timer so that each timer is triggered in a new thread. If this isn't done, the timer will try to interrupt an arbitrary thread to run it's function which, put plainly, is unreliable. Another alternative would be to have a dedicated thread for sending reminders and pass that as the argument to :thread but we'll take the easy way out this time. Add the following code to the end of reminder.lisp. Here are links to my versions of the files: messaging, reminder.

(defgeneric schedule (reminder)
  (:documentation "Schedule the reminder to be sent at the time the user requested."))

(defmethod schedule ((reminder reminder))
  (let ((secs-until-reminder (round (timestamp-difference (reminder-at reminder) (now)))))
    (trivial-timers:schedule-timer
     (trivial-timers:make-timer (lambda ()
                                  (send-and-delete reminder)) :thread t)
     secs-until-reminder)))

Next Time...

This article got long. It may be too high-level for some, too low-level for others and too wordy for everybody. In addition, this isn't the most thrilling software ever constructed. Next time we'll get into more Weblocks specifics and work on the frontend to get the form and a jQuery calendar up and going. Please let me know if there are questions I can answer, things you'd like covered in more depth or other thoughts on how to improve this series. Thanks for reading.

Albums of the year thoughts...

posted on 2010-11-14 20:21:27

I've been enjoying a lot of music of late and since the year is coming to an end thought it might be fun to try and think about what my Top 5 Favorite Albums I've found this year are. Obviously then this is about personal preference rather than some supposedly objective notion of quality. Hopefully I'll write a follow up piece in the first week or so of 2011 with my final thoughts/judgments.

Last.fm has some strong data-based opinions on this but they're also wrong about some things. For example, at some point I managed to play Bibio's album or a song or two off of it but then forgot and had to leave to do something. Except my player was on repeat...and scrobbling. It kind of makes me wish that last.fm stopped paying attention to scrobbles if the same track had already been scrobbled 10 or more times. Or exposed an option to enable such behavior.

Shoo-ins include:

• Laura Veirs - July Flame (This record is staggeringly beautiful and makes me wonder how I had never heard of Laura Veirs before. I find myself convinced that everyone everywhere should enjoy this album. It sounds that gorgeous.)


• The Morning Benders - Big Echo (It's insanely listenable. I just put it on and catch myself nodding my head regularly.)


• Metric - Fantasies (I know, it's from 2009. I just discovered it. Deal. Need to move? This is your album. Infectious.)

Other possibilities include:

• Jay-Z - The Blueprint 3 (The first half really wants to win, the second half not so much.)


• The National - High Violet (It's uniformly gorgeous but too dark to be listened to as much as I'd like...)


• The Bloody Beetroots - Romborama (I did listen to this a decent amount but I just don't love it that much.)


• Local Natives - Gorilla Manor (For a random debut this was quite impressive but my listening tapered off after a bit and I haven't gotten back to it quite yet.)


• Band of Horses - Infinite Arms (I enjoyed this album and still listen to it some but it's nowhere near as good as their previous two albums. And to be fair, I spent the first half of this year discovering and falling in love with those previous two albums so maybe they'll sneak into my final list.)


• Tame Impala - Innerspeaker (Another delightful album that my listening has tapered off for.)


• Teebs - Ardour (I've just started listening to this but it's *very* enjoyable. We'll see how long it keeps up.)


• White Rabbits - It's Frightening (I listened to this a lot early this year but it trailed off and it's doubtful it will make it on my list though it is a solid album.)

Albums I expected to be shoo-ins that really weren't include:

(NOTE:  Being on this list doesn't mean the album is not in the running. It means I expected it to be a shoo-in.)

• Gorillaz - Plastic Beach (After Demon Days I was prepared for an incredible album. I was thoroughly underwhelmed by Plastic Beach though think Empire Ants is one of the better songs Gorillaz have produced.)


• Massive Attack - Heligoland (This was much closer to my expectations after a few listens than Plastic Beach but I'd still put on Blue Lines or Mezzanine long before Heligoland. Hopefully the Burial remixes will come to fruition. To be fair I spent half the year waking up to Paradise Circus. It's that good.)


• Four Tet - There Is Love In You (This is a very good album but I don't listen to it that much. Whether that's because of how minimal it is or not I'm not sure.)


• Broken Social Scene - Forgiveness Rock Record (This wasn't as good as I was hoping but was more in the realm of Heligoland level disappointment than Plastic Beach. Meaning it's good it just doesn't measure up to previous efforts. Really, BSS will probably never top You Forgot It In People for me. World Sick, Sweetest Kill and All to All are insanely good songs though.)


• Two Door Cinema Club - Tourist History (I got really hooked on a number of early singles and EPs by this band after discovering them through Bryan O'Sullivan's twitter feed. They're wildly good and infectious pop. Good high-energy music. Hell, they were the soundtrack for the first half of the year along with lots of Band of Horses. But they reworked the singles on the EPs and I think the new versions are worse. This is a really good album but not quite a shoo-in. And it really should've been. Keep your eyes on these kids.)

I was hoping I'd find an analog to last year's discovery of Jon Hopkins' Insides or Ametsub's Nothings of the North but I haven't yet. If you know of something along those lines I should be hearing, please do let me know. Granted, since I discovered those albums late last year/early this year they could make it in my final list but we'll just have to wait and see. All for now.

The Spirit of Lisp (some young kid's impression)

posted on 2010-11-12 20:13:28

Disclaimer: I am not speaking for the CL community. Hell, depending on who you ask there *is* no Common Lisp community. If you're an old hat lisper, this article will probably make you groan, scratch your beard, kick something and mumble something like "Lisp just smells funny, News at 11". I'm posting it because I haven't put the words down before and the thoughts bring me joy. But *this is not news*. Because news is at 11. If you're a not-yet (common) lisper who has some interest in the language, my hope is that if you show up on #lisp or otherwise begin investigating or playing with lisp, you'll arrive with a slightly more informed perspective.

I recently read something on Zach Beane aka xach's blog that made me quite happy. He was posting about this year's International Lisp Conference and discussed (among other things) the low attendance this year and some possible causes of that. But then he went on to write this wonderful bit,

"I really like getting together in space and time with other Lispers. An ideal future Lisp conference for me, personally, would ... attract hundreds of cheerful and enthusiastic Lisp nerds...Navel-gazing and nostalgia would be at a minimum. People would talk about what they're doing today and what exciting things they plan for the future. Everyone would get together at dinner and swap stories about Lisp, life, and whatever else came to mind.

I know people are doing fun stuff with Lisp because I talk to them every day about it online. It would be pretty special to talk to them for a few days about it face-to-face."

...which in part led me to tweet the following just because I thought it embodied some things I really love about the lisp community:

"(loop (awhen (build-something) (release it))) ;; Wake up #lisp ers. Your time is now. This message brought to you in part by #quicklisp"

Nikodemus Siivola nailed this too at some point with the quote, "Schemer: “Buddha is small, clean, and serious.” Lispnik: “Buddha is big, has hairy armpits, and laughs.” Scott Fahlman's statement that "Common Lisp is politics, not art." seems similarly indicative of this in some ways also.






Generally, the whole conversation devolves. The parties are coming from totally different points of inquiry. But whether the visitors are "flamed so hard they die" or gently dealt with until agreements are reached things end pretty quickly and everyone goes on about their day. Paul Snively calls lisp the "cockroach of programming languages" and I'm not sure if he means Common Lisp or just the Lisp family genes. But when people ask if lisp is dead or "shouldn't I just use/study language X?" I'm relieved and pleased that we're too busy having fun and building things to worry about it. Who cares what language wins tomorrow? This language works today, we're using it and when we see other languages with something we need, we grab it.

To me this is something refreshing about the lisp community that isn't internally or externally recognized quite enough. Which isn't to say that we should go around beating our chests and talking about what rock stars or great programmers we are. I'm certainly not one. I remain a wet behind the ears programmer. I'm not writing as much code as I should and as a consequence still have little useful stuff to release. But I've been privileged to watch and try to help Will Halliburton with a lisp-powered startup and work with Leslie Polzer on Weblocks and Paktahn.

Really it's most likely that there's so much more noise than signal regarding "lisp" and often so little clarity as to whether scheme, lisp or genetically 'lispy' languages are being discussed on online forums that the public image about Common Lisp is horribly out of whack. Hell, I've said terrible, stupid, ridiculous things about Common Lisp in the past. Why? I hadn't seen the community, I hadn't seen the language, I didn't understand *what* it was. And maybe we can't change that or it's just not worth the effort. The people who have heard something from x, who heard it from y, who heard it from z will keep repeating old wives' tales forever. But here's my attempt at getting down why this language is in no danger of dying anytime soon:

We're all just having fun building things. And if that sounds like something you'd like to do, please come in, visit #lisp and ask us questions. Message me personally if you want, I'm on freenode as redline6561. I promise the water is fine. Just don't ask if this is the right way to use your time. Figure that out in advance. As far as I'm concerned, between great open source implementations like SBCL and CCL, great editing solutions like SLIME for emacs, Slimv for vim and the new Textmate bundle and easy access to libraries through quicklisp, there's never been a better time.

CL Web Primer Progress Report

posted on 2010-11-07 21:20:29

Wow. It's been way too long since I've written about this. Naturally, software takes longer than expected. I also helped Leslie test features that were in beta so migrating to those and working out their kinks slowed things down a little.

But long story short, the next entry in the CL Web Primer series should be coming very soon. 7 days at the latest. If you just want to read the code, it's at http://github.com/redline6561/clockwork.

Defining a jQueryUI Datepicker presentation, 38 lines.
Defining a simple text messaging system, 37 lines.
Defining a reminder class and methods to schedule the reminder to be sent, send it and delete it, 47 lines.
Defining a form for users to fill out, a list of timezones and validation functions to ensure the input is good, 109 lines.
Putting it all together, 27 lines.
Learning about web programming (I know, giving myself too much credit there...) with Common Lisp and Weblocks? Priceless.

The site is still quite ugly and needs a lot of work and a few more features but at least it's functional now.
Until next time...

This blog covers 2015, Books, Butler, C, Dad, Discrete Math, Displays, Education, Erlang, Essay, Gaming, Gapingvoid, HTDP, Hardware, IP Law, LISP, Lecture, Lessig, Linkpost, Linux, Lists, MPAA, Milosz, Music, Neruda, Open Source, Operating Systems, Personal, Pics, Poetry, Programming, Programming Languages, Project Euler, Quotes, Reddit, SICP, Self-Learning, Uncategorized, Webcomic, XKCD, Xmas, \"Real World\", adulthood, apple, career, careers, choices, clones, coleslaw, consumption, creation, emulation, fqa, games, goals, haltandcatchfire, heroes, injustice, ironyard, linux, lisp, lists, math, melee, metapost, milosz, music, pandemic, personal, poetry, productivity, professional, programming, ragequit, recreation, reflection, research, rip, strangeloop, vacation, work, year-in-review

View content from 2024-06, 2024-03, 2024-01, 2023-12, 2023-07, 2023-02, 2022-12, 2022-06, 2022-04, 2022-03, 2022-01, 2021-12, 2021-08, 2021-03, 2020-04, 2020-02, 2020-01, 2018-08, 2018-07, 2017-09, 2017-07, 2015-09, 2015-05, 2015-03, 2015-02, 2015-01, 2014-11, 2014-09, 2014-07, 2014-05, 2014-01, 2013-10, 2013-09, 2013-07, 2013-06, 2013-05, 2013-04, 2013-03, 2013-01, 2012-12, 2012-10, 2012-09, 2012-08, 2012-06, 2012-05, 2012-04, 2012-03, 2012-01, 2011-10, 2011-09, 2011-08, 2011-07, 2011-06, 2011-05, 2011-04, 2011-02, 2011-01, 2010-11, 2010-10, 2010-09, 2010-08, 2010-07, 2010-05, 2010-04, 2010-03, 2010-02, 2010-01, 2009-12, 2009-11, 2009-10, 2009-09, 2009-08, 2009-07, 2009-06, 2009-05, 2009-04, 2009-03, 2009-02, 2009-01, 2008-12, 2008-11, 2008-10, 2008-09, 2008-08, 2008-07, 2008-06, 2008-05, 2008-04, 2008-03, 2008-02, 2008-01, 2007-12, 2007-11, 2007-10, 2007-09, 2007-08, 2007-07, 2007-06, 2007-05