March 6, 2015

From Pedestal routes to beautiful documentation

If you're doing server side Clojure I'm sure you know and use Prismatic/schema. If you don't, you probably should. At Juxt we love it and we use it exstensively. The performance hit is not trascurable, but you can always turn checks off for your production environment while reaping the benefits in dev and testing.

We tacitlly agreed to always have a schema whenever we are doing I/O. Schema checks on the way in make user input validation and coercion to known types much easier and declarative, and while the error reporting is not user friendly out of the box it doesn't take much effort to customise it.

Server responses are normally made out of data aggregated from various sources: request parameters, user session, databases, external services, you name it. Because Clojure makes working with data so easy we found it's not uncommond to leave in the response some unnecessary information assoc'd deep down a nested map. Or to forget to convert all map keys to a uniform case (wich one do you use in your JSONs: snake_case or kebab-case?). Schema validation on our way out ensures we don't break the data contract with our client apps.

So when the time came to write an extensive documentation for all our server endpoints we though: why not just generate it from our well defined schemas? That must have been the same thinking the guys from Metosin had when they wrote ring-swagger, a library that converts schemas to a JSON spec understood by Swagger. I leave out to you to find out more about Swagger, but just think that you can easily generate documentation for your server that looks like this.

Two of the most popular web frameworks already have specific libraries that convert annotated endpoints to swagger schemas: Compojure and fnhouse. I decided to write a library to takes care of Pedestal too.

Embrace the interceptor

One thing particularly interesting about pedestal-swagger is that it leverages Pedestal's interceptor mental model for documentation. With Pedestal is common to split your endpoint logic in reusable interceptors. They look a lot like middlewares except they are path specific. For example if you have a resource that offer GET, PUT and DELETE methods you usually want to check that that resource exists first and load it up in memory, or return a 404 if it doesn't. Or you want to protect a path with some kind of authentication by placing an interceptor at its root and return 403 if Forbidden. (Example taken from my previous post)

(defbefore load-order-from-db
 [{:keys [request] :as context}]
 (if-let [order (get-order-from-db request)]
  (assoc-in context [:request :order] order)
  (-> context terminate (assoc-in [:response] {:status 404}))

(defbefore basic-auth
 [{:keys [request] :as context}]
 (if-let [{:keys [username password] :as auth} (check-basic-auth request)]
  (assoc-in context [:request :user] auth)
  (-> context terminate (assoc-in [:response] {:status 403}))

(defroutes routes
 [[["/orders/:id" ^:interceptors [load-order-from-db]
      {:get get-order
       :delete delete-order
       :put update-order}]
    ["/secure" ^:interceptors [basic-auth]
      ["/path1" {:get secure-1}]
      ["/path2" {:delete secure-2}]]]])

With pedestal-swagger you can attach metadata to single interceptors so both the documentation and the schemas are inherited by all the endpoints under that path.

(swagger/defbefore load-order-from-db
 {:parameters {:path {:id schema/Int}}
  :responses {404 {:description "Not found"}}}
 [{:keys [request] :as context}]
 (if-let [order (get-order-from-db request)]
  (assoc-in context [:request :order] order)
  (-> context terminate (assoc-in [:response] {:status 404}))

(swagger/defbefore basic-auth
 {:description "Requires Basic Auth"
  :parameters {:header {:basic-auth schema/Str}}
  :responses {403 {:description "Forbidden"}}}
 [{:keys [request] :as context}]
 (if-let [{:keys [username password] :as auth} (check-basic-auth request)]
  (assoc-in context [:request :user] auth)
  (-> context terminate (assoc-in [:response] {:status 403}))

That ensures that the behaviour and its documentation sit close together. When you change one you don't need to wander around your code for occurencies of the other.

Another cool feature interceptors enable is flexible coercion and validation logic. Instead of rebinding some dynamic vars coercion and validation is provided as two first class interceptors (swagger/coerce-params and swagger/validate-reponse) that can be included in your routes. If you don't like their default behaviour you can simply roll your own and place them at the root of your application. Or you can have different behaviours on different paths. And you can decide wether to include them in your test routes or not.

Data driven or go home

I'm afraid readers will think I'm talking about some other language if I don't mention data at least once. Yes this is Clojure. Yes let's talk about data.

Pedestal is data driven. The configuration needed to create a service, server or servlet is contained in one map. Because of this decision, extending Pedestal is infinitely easier than something like Compojure. All pedestal-swagger has to do is walk thought that map to collect the information it needs and massage it in a format Swagger understands. Compare this to the pain of converting a macro dsl to another macro dsl while inspecting opaque functions. Not as much fun in my opinion. (Big props to compojure-api for unlocking that achievement nonetheless).

Having the configuration all in one place gives you a good hint about where to store your additional layer on top of it. An earlier implementation of pedestal-swagger stored the computed result in a separate var that was overriden at compile time. That plan backfired pretty soon: tests where hard to write, reloading the repl yielded odd results, the library was harder to include safely in other projects.

The solution to avoid place oriented programming is to store all the configuration in one place: in this case, the route table. Because the interceptors are passed the route table as an argument I was able to retrieve the generated documentation and use it for display/coercion/validation.

Things that might explode

Now, Swagger 2.0 release was a complicate one. It was first announced 6 months ago, but related tools like Swagger UI and Swagger Codegen are just now starting to look useful. The guys at metosin have done a great job in keeping their library up to date with the latest swagger updates, but bear in mind: when you're using pedestal-swagger you're using beta code that depends on beta code that depends on beta code. Do your own maths.

That being said the library seems to work well and every tool in the chain is pretty extensible if you like to spend some time with it. I appreciate your feedback.

Tags: pedestal swagger