December 15, 2014

Give Pedestal another chance

For a community that is so eager to embrace new technologies, it surprises me how few Clojure developers have adopted Pedestal as their default web stack.

Ring + Compojure seems to be the go to choice when building a web server, throwing in the occasional Liberator or Bidi.

Mind that there's actually nothing wrong about it. Ring applications are extremely compact and easy to understand. For those unfamiliar with it, Ring is based on a simple idea: your server is a function. This function (a Ring handler) takes a request map and returns a response map.

(def request
  {:uri "/hello"
   :query-params {:name "Frankie"}})

(defn handler [request]
  {:status 200 :body (str "Hello, " (get-in request [:query-params :name]))})

(handler request)
;; => {:status 200 :body "Hello, Frankie"}
You can build your stack on top of Ring composing the server function with other functions. For example Compojure wraps the server function with a routing function that inspect the requested path and dispatches it to the appropriate Ring handler.
(defn welcome [request]
  {:status 200 :body (str "Welcome, " (get-in request [:query-params :name]))})

(defn goodbye [request]
  {:status 303 :headers {"Location" "/"})

(defroutes app
  (GET "/login" request (welcome request))
  (GET "/logout" request (goodbye request)))

(app {:uri "/login"
      :query-params {:name "Frankie"}})
;; => {:status 200 :body "Welcome, Frankie"}

(app {:uri "/logout"})
;; => {:status 303 :headers {"Location" "/"}

Liberator extends the Ring handler concept with an HTPP State Machine: it calls a series of functions in its decision tree with the request map to finally generate the response.

(defresource example
  :handle-ok "This is ok"
  :handle-not-found (fn [_] "Ops.")
  :authorized? (fn [_] ...))

You can start to see how things are getting a little bit complicated by now, and it's not because any tool in particular is complex (they're small focused, simple libraries that compose well with each other). It's because at the bottom of the stack there is a function.

If you give me a function, you're limiting my options. I can call it, I can compose it, I can wrap it, but that's pretty much all. I cannot inspect which functions are wrapped inside it, I cannot remove or add behaviour for specific cases, I cannot distribute the execution over several threads, I cannot easily debug it (have you tried debugging Ring middlewares?). Moreover, since the lower level libraries have functions as a foundation, it is normal for higher lever libraries to build their DSLs with macros (because you need to rearrange function calls), and once the next higher level library is based on a library which exposes its api via macro, you're in for some troubles.

Data beats function composition

Pedestal adresses these problems by having at its foundation a data structure: the route table. A route table is a list of routes (identified by name, path regex and http method) that fully describes your web service. From this route table you can always generate a server function, but that's a step that you do the moment you actually need to run the server. The route table is what you pass around your application, is how you inspect and debug you web service, is how you reason about it.

[{:route-name "foo" :path "/foo" :method :head ... :interceptors [...]}
 {:route-name "bar" :path "/bar" :method :path ... :interceptors [...]}]

Each route contains a second data structure: the interceptor list. As the route table simplifies reasoning about routing, the interceptor list simplifies reasoning about your middleware. An interceptor is like a middleware function, only it operates on a context map like:

(def context
 {:request  {...} ;; the incoming request
  :response {...} ;; the response built so far
  :route    {...} ;; the selected route
  ...})

Incerceptors are called one after the other when the request comes in, and again in reverse order when the response comes out. Since it's a list and not a function, you can manipulate the call chain like you manipulate any other data structure. And since the execution happens in stages you can pause a thread that is serving the response, store the context map, and resume the interceptor chain later on.

By default interceptors are inherited by children routes, so you can define fine grained behaviour for a certain list of resources:

;; These are two interceptors executed only when the request comes in.
;; 'terminate' drops the remaining interceptors in the execution list and start traversing it backwards.

(defbefore load-order-from-db
 [{:keys [request] :as context}]
 (if-let [order (get-order-from-db request)]
  (assoc-in context [:request :order] order)
  (-> context terminate (assoc-in [:response] {:status 404}))

(defbefore basic-auth
 [{:keys [request] :as context}]
 (if-let [{:keys [username password] :as auth} (check-basic-auth request)]
  (assoc-in context [:request :user] auth)
  (-> context terminate (assoc-in [:response] {:status 403}))

;; This is the concise way to specify a route table. Is itself a data structure (instead of a macro).
(def routes
 [[["/orders/:id" ^:interceptors [load-order-from-db]
      {:get get-order
       :delete delete-order
       :put update-order}]
    ["/secure" ^:interceptors [basic-auth]
      ["/path1" {:get secure-1}]
      ["/path2" {:delete secure-2}]]]])

All handlers under "/orders/:id" are invoked after the order is loaded from the database, and the interceptor can short circuit returning 404 before even reaching the handlers.In the same way, all handlers under "/secure" are served only after basic auth is checked.

If you give me a route table with interceptor lists then I can do all sort of things with it. I can manipulate the data structure differently for dev/production environments. I can easily generate automatic documentation with a library on top of it. I can reuse behaviour defined elsewhere. I can decide at any stage to short circuit the excecution without setting up a hacky throw/catch among middlewares.

I don't think the Pedestal documentation does a good job of conveying how empowering is to move away from function chains and embrace data structures at the bottom. The concepts of route table and interceptor list seem complicated at first, while they are in the end extremely easy to grasp (a feeeling that I experienced many times while learning Clojure). I don't think this blog post does a better job in explaining them but I hope it inspires you to try pedestal past the tutorial and actually use it to reimplement an existing web project that you might have, so you can see how things fall beautifully into place.

Tags: pedestal