Composability and Compojure

2012-02-23T23:57:00+05:30

Caveat: This post needs some rudimentary knowledge of Compojure. Compojure is a web framework for Clojure, and if you don't understand what that means, then you should probably head over to the Compojure docs. Compojure exposes us to a beautifully composable abstraction, and this post is an attempt to show why that is a great thing.

In order to understand the Compojure framework, let us write a small cello world app. The code snippets in this post are a means to explain a concept, they may not work as-is. The full, working code is available at this gist. Okay then, let's get started.

Here is what the basic routes function would look like:

basic_routes.clj

(cc/defroutes main-routes
  (cc/GET "/" [] (fn [req]
                   (rur/response "<h1>Cello World</h1>")))
  (cc/GET "/bye/" [] (fn [req]
                      (rur/response "<h1>Goodbye World</h1>")))
  (route/not-found "<h1>Page not found</h1>"))

Compojure uses Ring to handle requests and responses. These terms (request/response) don't mean 'objects' (to those of you from the OOP world) of any kind, they are just hash-maps used to represent the idea. They are data, and as such, can be manipulated in any way we want. Ring has a simple philosophy: write functions which accept a request and return a response. Such functions are called handlers.

The cc/GET helper macro - and it's ilk - asks the user for a request-method (specified by the GET in the name cc/GET), a route (string representing uri), and a handler(H1). The macro becomes a handler (H2) which returns the result of (H1 req) if the route and the method of the incoming request match the specified route and request-method, otherwise it returns nil. Wow, that was a mouthful, wasn't it?. Stop snickering and saying "That's what she said". So anyway, this is our first introduction to composability in Compojure. We have a macro that takes a handler and gives us another handler. As long as we are dealing in handlers, composability allows us to ignore any implementation complexity.

The cc/defroutes macro takes a name and a list of handlers and returns a handler (H3) which runs all the handlers in the list on the request until one of them returns a non-nil value, else it returns nil. Finally, it binds the name to the handler so that we can call it. Simple, isn't it?

So if you think you've understood it so far, tell me if adding this route to our main routes will work or not:

composability-one-oh-one.clj

(cc/GET "/hello*" [] (cc/defroutes hello-routes
                         (cc/GET "/hello/name/" []
                                 (fn [req]
                                   (rur/response "<h1>Cello Vedang</h1>")))
                         (cc/GET "/hello/city/" []
                                 (fn [req]
                                   (rur/response "<h1>Cello from Pune!</h1>")))))

Well, yes! cc/defroutes gives us a handler¹, and that's really all cc/GET cares about!

Having an abstraction of this form allows us to do many things easily, knowing that stuff just works. For example, let us write some middleware. What is middleware? Middleware modifies the incoming request or outgoing response in some way that makes us happy. How does this fit into our abstraction? - As a function that takes a handler(H1) and, wait for it, returns another handler(H2). Boom! Confused? Here is what a middleware function looks like:

middleware.clj

(defn verify-secret
  "Verify that secret-key has been sent as a parameter s in the request"
  [handler]
  (fn [request]
    (if (= "s=please" (:query-string request))
      (handler request)
      {:status 403
       :body "You don't know the secret word"})))

This function gets a handler(H1). It doesn't know or care what that handler is going to do to the request. It returns a new handler(H2) which does the following: it checks to see if the incoming request knows that the secret word s is "please". If it does, great. Execute H1 on the request and call it a day. Otherwise, return a nil - meaning the request in not valid. Now H2 could go through as many other middleware functions as we want, all of them agnostic of any other middleware functions. Each middleware will return a modified handler(H3, H4, ... Hn), and we will run the final handler on the request.

Here is what the final code would look like²:

compojure-middleware.clj

;; Define some Hello routes
(cc/defroutes hello-routes
  (cc/GET "/hello/name/" []
          (fn [req]
            (rur/response "<h1>Cello Vedang</h1>")))
  (cc/GET "/hello/city/" []
          (fn [req]
            (rur/response "<h1>Cello from Pune!</h1>"))))
;; Our main routes function.
(cc/defroutes main-routes*
  (cc/GET "/" [] (fn [req]
                   (rur/response "<h1>Cello World</h1>")))
  (cc/GET "/bye/" [] (fn [req]
                      (rur/response "<h1>Goodbye World</h1>")))
  (cc/GET "/hello*" [] hello-routes)
  (route/not-found "<h1>Page not found</h1>"))
;; A middleware function
(defn verify-secret
  "Verify that secret-key has been sent as a parameter s in the request"
  [handler]
  (fn [request]
    (if (= "s=please" (:query-string request))
      (handler request)
      {:status 403
       :body "You don't know the secret word"})))
;;; Wrap my main routes in middleware
(def main-routes (-> #'main-routes*
                     verify-secret))
;;; Start app with main routes
(run-jetty #'main-routes {:port port :join? false})

main-routes* is a handler which matches the incoming uris to ones we support, verify-secret will make sure that the incoming requests know the secret word. We can go a really long way with functions that take a request and return a response. Compojure gives us a great DSL to deal with the web. It's composability facilitates building elegant systems and frameworks.

Actually, we got lucky in this case. `cc/defroutes` is a macro. When we say `(cc/defroutes name & handlers)` the code is replaced to become `(def name handlerfn)`. Luckily for us though, def returns the variable which was just defined, and it works out okay in the end. The aim was to show composability in action, not to espouse a coding style. Never do this in actual code. ↩
Apparently, we can't get line spacing in the octopress codeblock ↩

Switch Statements in Python

2011-08-07T13:21:00+05:30

Today I went through some old Python code and noticed this pattern through out:

messy-ifs.py

response, data = somefunc()
if response == "this":
    do_this_with(data)
elif response == "that":
    do_that_with(data)
elif response == "huh":
    duh(data)
# lots more elifs.
else:
    prevent_horrible_crash(data)

This code should ideally have been a switch-case, but Python does not support a switch statement. Proponents of OOP believe that switch is bad - second only to goto. This is not strictly true - both goto and switch can be used elegantly and with great effect. Goto, for example, is great for undoing stacked changes and switch's fall-through behavior allows nicely for 'do things according to the stage I'm at'. However, if you're doing OOP, consider using polymorphism instead.

I'm not doing OOP, so polymorphism does not apply to me, but I was looking for a way to optimize this code. I was on a code-cleanup spree, in the "flow", and I thought, "Why don't I use maps to do this?". Brilliant! It was a really neat idea. Later I found out that this is the accepted way of doing switch-case in Python and I'd done nothing special. So anyway, the code now looked like this:

maps-as-switch.py

response_map = {"this": do_this_with,
                "that": do_that_with,
                "huh": duh}
response_map.get(response, prevent_horrible_crash)(data)

And that would have been that, had I not suddenly developed a conscience. I had replaced perfectly working, mostly readable code with some other code. What if my map solution was slower? What if it was much slower? I'd done a sizable amount of refactoring, and I did not relish throwing it away. I needed to test my solution, so I wrote some sample code:

from timeit import Timer from random import randint def switch_if(): value = randint(1, 10) if value == 1: return '1' elif value == 2: return '2' elif value == 3: return '3' elif value == 4: return '4' elif value == 5: return '5' elif value == 6: return '6' elif value == 7: return '7' elif value == 8: return '8' elif value == 9: return '9' else: return '10' def switch_map(): value = randint(1, 10) smap = {1: '1', 2: '2', 3: '3', 4: '4', 5: '5', 6: '6', 7: '7', 8: '8', 9: '9', 10: '10'} return smap[value] t = Timer(setup='from __main__ import switch_if', stmt='switch_if()') print t.timeit() t = Timer(setup='from __main__ import switch_map', stmt='switch_map()') print t.timeit()

The results were disheartening. My replacement code was slower - though only just.

$ python switch-speed.py
if - 2.08906793594
map - 2.88215684891

I tweaked the code a little and moved the creation of the map outside of the switch_map function. Python can access local variables faster than it can access global variables, but I figured a global dict would still beat the cost of creating the dict every time.

$ python switch-speed.py
if -  2.08955693245
map -  2.00381493568

I breathed a sigh of relief. There was a lesson to be learnt here - test/profile before you optimize!

References and articles:

More information about switch statements smell - c2.com
Goto is not evil - kerneltrap.org

Why I Quit StumpWM

2011-08-04T13:41:00+05:30

After using StumpWM for nearly a year, I'm finally quitting and moving back to Gnome. I came to love StumpWM's minimalistic approach, and this isn't an easy decision. So why am I doing it?

Pinky Hell

I am (was) a heavy user of Emacs, Tmux and StumpWM. These programs rely heavily on the Ctrl key. This means that the Ctrl key and my left pinky finger are pretty much destroyed at this point. With the spectre of RSI looming over my head, StumpWM is the easiest tool to get rid off. I might have less screen-space now, but my fingers will thank me for it.
What you gain in Functionality, you lose in Integration

StumpWM is infinitely configurable. If you have the patience, you can get it to do things that are simply impossible with other Windows Managers. For example, there is a mpd.lisp module inside the contrib directory which converts StumpWM into a MPD client. A full-fledged client. This means that all music operations are entirely unobtrusive and always at your beck and call. MPD with StumpWM has without a doubt been the best music setup I've ever used.

However, I missed Gnome's integration with Ubuntu - Ubuntu One, Gwibber, Pidgin, event notifications. Even if I could configure StumpWM for these things, there was too much effort involved.
Dead processes, High temperatures

Disclaimer : Everything I'm about to say now is conjecture. There is no proof that StumpWM is responsible for any of this.

At 5 days uptime, my system load was touching 2, and my system temperature was at 65 degrees C. KK had a similar number of programs running on his system - this is a guy who uses full-blown compiz effects - his system temperature was 36 degrees and load average was around 1.15. Furthermore, he had a 48 day uptime. KK gleefully told me that it had to be StumpWM that was the problem! He was kidding, but some inquiry around the office showed that my laptop was much worse than the other guys on Ubuntu.

I also found that Chrome was leaving behind loads of defunct processes on my machine. There were 41 defunct chromium-browser processes when I decided to reboot the laptop and do some monitoring. Within an hour of the reboot, I had 7 defunct processes, CPU temperature was at 45 degrees and the load average was hovering around 1.8. That was the proverbial straw that broke the camel's back. I switched back to Gnome. Later in the day, I thought about actually investigating the behavior of my laptop, but it's been behaving well on Gnome. As I write, I have a day's uptime, a load average of 1.03 and CPU temperature at 37.5 degrees C. Unscientific, but fine by me.

How I got over the Withdrawal Symptoms:

Of all the StumpWM features, there was just one that I simply could not do without - 'run-or-raise'. On StumpWM, you can define hot-keys to behave as follows:

Launch the program if it is not running
Move focus to the program if it is already running.

I use one Emacs instance, one terminal instance (tmux'ed) and one Chrome instance. I need to be able to access them with minimal friction. Turns out, a little hackery on compiz is all we need - compiz-send.py is a nifty little script that sends messages to DBus and interacts with compiz. I can use it along with xdtool to implement my run-or-raise behavior. Here is a look at my run-or-raise-emacs.sh script:

#!/bin/bash # get id of emacs win=$(xdotool search --onlyvisible --class emacs | head -1) if [ "x$win" = x ]; then # Emacs is not running # switch to viewport 1, which is my coding viewport ~/incoming-src/dotfiles/scripts/compiz-send.py vpswitch switch_to_1_key gxmessage -center -timeout 2 -nofocus -buttons "" -borderless "loading emacs..."& # start Emacs exec /usr/local/bin/emacs & else # Emacs is already running... if [ $win = $(xdotool getwindowfocus) ]; then # ...and focused # do nothing sleep 0.2; else # switch to it xdotool windowactivate $win fi fi

You can download compiz-send from here.

So there you have it. At the moment, life is back to Gnome+Compiz. If you know any nifty hacks for Gnome+Compiz, please let me know in the comments.

References:

wiki.compiz.org

Reboot.

2011-07-30T14:24:00+05:30

Once upon a time I had a blog. Then one day I ran out of motivation, inclination, desire, drive, whatnot to post to that blog. I never really ran out of content though. At least, I think not.

I worried that I'd drown this world in my drivel, but lately I've come to realise that no one gives a shit. So I'm going to write for the right reasons: Because it makes me happy. Stick around, and I'll try not to bore you. Or leave. I don't give a shit.