Catacumba - Web toolkit for Clojure.

Introduction

Catacumba is an asynchronous or non-blocking web toolkit for Clojure built on top of ratpack and netty drawing inspiration from ring, pedestal and ratpack.

Note	Project Maturity: Since catacumba is a young project there may be some API breakage. Rationale: You can read the rationale behind this project here.

Quick Start

This section intends to explain how to get catacumba up and running.

Install

The simplest way to use catacumba in a clojure project is by including it in the dependency vector on your project.clj file:

[funcool/catacumba "2.2.1"]

Note	Catacumba will only run with JDK8 and Clojure >= 1.7.

Handlers

The handler consists of a function that accepts a "context" as it’s first parameter and returns something renderable. Let’s see an example:

(defn example-handler
  [context]
  "Hello World")

It looks very similar to ring handler with two main differences:

instead of request it receives a context that works like request but with more responsibilities (explained in other sections).
returns a string instead of a hash-map (it is also allowed but is not mandatory, also explained later).

Routing

Now knowing how we can define handlers, the next step is define a route (http endpoint) for our handler. That can be done with routes function:

(require '[catacumba.core :as ct])

(def app
  (ct/routes [[:all "" example-handler]
              [:get "foobar" example-handler]]))

The routes function receives a vector of ordered entry points for our handlers. In this example we have defined two routes for the same handler (just for demonstration purposes):

The first entry defines a / route for all kind of requests and the second entry defines a /foobar route only for GET requests for the same handler.

You can read a complete documentation about catacumba’s routing here.

Run the server

For run the previously defined handler, just use the run-server function:

(ct/run-server app {:port 3030})

Tip	The `run-server` function doesn’t block so you can execute it in a repl without problems.

You can read more about all available options that you can pass to run-server function here.

Put all together

This is what the complete source code of the example looks like:

(ns exampleapp.core
  (:require [catacumba.core :as ct])
  (:gen-class))

(defn example-handler
  [context]
  "Hello World")

(def app
  (ct/routes [[:all "" example-handler]
              [:get "foobar" example-handler]]))

(defn -main
  [& args]
  (ct/run-server app {:port 3030}))

Catacumba also comes with a little collection of Examples that may help you setup your first project.

User Guide

This section intends explain all the different parts of catacumba and how they work together.

Handlers

Is the fundamental building block in the catacumba toolkit and has two main types:

Ending: Handlers that process a request and return a response (usually named controller in other web frameworks or toolkits).
Middle: Handlers that does some logic but does not return any response ( delegating that task to other handler) (usually named as middleware or decorator).

The both handler types are defined in term of functions and looks identically, the principal change is the responsibility. Let see an example:

An ending handler example.

(defn sample-ending-handler
  [context]
  "Hello World")

An middle handler example.

(defn sample-middle-handler
  [context]
  (println "hello world")
  (ct/delegate))

As you have seen, the examples until now are always returning a simple string that in fact is not very useful in real-world use cases. The great news here is that return values are handled using open polymorphic abstractions such are protocols.

This means that you can return anything that catacumbla already has implementation for it or anything that yourself have implemented. Let see some examples:

Handler example that returns ring style response.

(defn some-handler
  [context]
  {:status 200
   :headers {}
   :body "Hello World"})

Handler example that returns catacumba’s builtin response type.

(require '[catacumba.http :as http])

(defn some-handler
  [context]
  (http/ok "Hello World"))

But this is not the end, if you want to know all the different kind of handlers and its return types, please take a look on Handler types section.

The middle handlers

The catacumba toolkit in request/response handling perspective behaves like a an asynchronous pipeline of handlers (when only one handler is attached it will be a pipeline of one unique element).

As you have observed in the previous example, the middle handler instead of returning a response, returns a result an opaque type that indicates to catacumba that this handler is not of ending type and forces to catacumba take the next handler from the pipeline and execute it. This step is done asynchronously. And so until ending handler is found and response is returned.

There are nothing especial, the opaque object that delegate function returns just implement appropriate protocol and if you don’t like the default behavior, you are free to implement your own.

This is a little introduction to the delegation process and how the middle handlers participates on it. For in depth understanding and how you can use it in your application, please read the handler delegation section.

Context

The second thing most important in the catacumba is the context. It can be considered a central part of both: IO and the control flow (will be explained in advanced section).

In other words, it can be considered as a combination of request and response, but in most cases you will use it like a request. It exposes the already familiat set of attributes: :method, :query, :path, :headers and :cookies. And all them are accessible with keyword lookup:

(:method context)
;; => :get

This is a reference table of request attributes are availeble under context:

Key Type Description Example

Key	Type	Description	Example
`:body`	`TypedData`	A object that represents a request body (not always awailable, see below).
`:method`	`Keyword`	A request method.	`:head`, `:trace`, `:get`, `:options`, `:put`, `:post`, `:patch` and `:delete`
`:query`	`String`	A raw string representation of the uri querystring.	`"foo=bar&baz=1"`
`:path`	`String`	A raw string representation of the uri path.	`/auth/client`
`:headers`	`PersistentMap`	A optionally multi value hash map of the request headers.	`{:host "funcool.org" :pragma "no-cache" …}`
`:cookies`	`PersistentMap`	A optionally multivalue hash map of request cookies (explained in details in its own section).
`:query-params`	`PersistentMap`	A optionally multivalue hash map of the parsed `:query` string.	`{:foo "bar" :bar 1}`

:body

TypedData

A object that represents a request body (not always awailable, see below).

:method

Keyword

A request method.

:head, :trace, :get, :options, :put, :post, :patch and :delete

:query

String

A raw string representation of the uri querystring.

"foo=bar&baz=1"

:path

String

A raw string representation of the uri path.

/auth/client

:headers

PersistentMap

A optionally multi value hash map of the request headers.

{:host "funcool.org" :pragma "no-cache" …}

:cookies

PersistentMap

A optionally multivalue hash map of request cookies (explained in details in its own section).

:query-params

PersistentMap

A optionally multivalue hash map of the parsed :query string.

{:foo "bar" :bar 1}

The only exception to the rule is the :body attribute, that by default does not comes. This is because this operation is delayed until is really needed and is done asynchronously, using get-body! function.

Nevertheless, in most cases you will prefer use a more high level and extensible abstraction that parses the body using it’s content type, see body parsing section for more information.

Note	In previous versions, body was always available as `:body` attribute in the context, although is not very efficient approach, you can return the previous behavior just using the `catacumba.handlers.parse/read-body` auxiliar handler on top of your route pipeline or as decorator.

Routing

In contrast to ring, catacumba is a toolkit for web development and offers builtin support for advanced routing that allows handlers chaining, partitioning, error handling, among other features.

Note	Catacumba has a polymorphic and extensible way to setup handlers, and routing is one of multiple possible implementations. Is completely optional and you can use any other routing library if you want.

Basic syntax

The routes in catacumba are defined using clojure data structures: vectors and keywords. Let’s see a little example of the aspect in a complete example:

(def routes
  (ct/routes [[:prefix "api"
               [:get "users" users-handler]]]))

The order of statements is very important because the routing in catacumba is a simple chain or pipeline. Each handler has the ability to delegate the request handling to the next handler in the pipeline.

This is a complete list of route directives that you can use a part of :get: :any (matches all routes, often used for add chain handlers), :post, :put, :patch and :delete.

Dispatch by method

In some circumstances you may want have different handlers depending on the HTTP method used for one concrete endpoint. You can do it in the following way:

(ct/routes [[:prefix "api/users"
             [:get list-users-handler]
             [:post create-users-handler]]])

This also can be done in this an other way:

(ct/routes [[:get "api/users" list-users-handler]
            [:post "api/users" create-users-handler]])

But is considered not idiomatic and the first example should be considered the right way to do it.

Note	Before, there was an other way to setup by method using the `:by-method` routing directive. It is now deprecated and will be removed in the next versions.

Routing params

catacumba's routing also allows to capture URL values encoded in the URL or as URL parameters using special symbols. For example, the path string "foo/:val" will match paths such as "foo/bar", "foo/123". The matched parameters are automatically populated to the context under the :route-params key:

(def article-detail
  [context]
  (let [id (get-in context [:route-params :id])]
    (http/ok (str "You have requested article with id=" id))))

(def app
  (ct/routes [[:get "articles/:id" article-detail]]))

Additionally to the basic token for representing URL parameters, catacumba also allows the use of regular expressions for delimiting the input or marking a URL token optional.

See the following table for all supported URL tokens:

Table 1. Supported url matching tokens
Path Type	Syntax	Route example	Matching url example
Literal	`foo`	`[:get "foo" handler]`	`/foo`
Mandatory	`:«token-name»`	`[:get "foo/:param" handler]`	`/foo/bar`
Optional	`:«token-name»?`	`[:get "foo/:param?" handler]`	`/foo` and `/foo/bar`
Mandatory & Regex	`:«token-name»:«regex»`	`[:get "foo/:id:\d+" handler]`	`/foo/2`
Optional & Regex	`:«token-name»?:«regex»`	`[:get "foo/:id?:\d+" handler]`	`/foo/2` and `/foo`

Routing chain

The chaining of handlers can be done in two different ways:

inline: providing more that one handler for concrete http method.
multiple routes: providing a "match all" handler at the start of prefix.

Chaining handlers inline follows this pattern:

(ct/routes [[:get "users" permission-check-handler get-users-handler]])

Additionally, you can setup "match all" handlers at the start of a routing definition and use them as interceptors:

(def routes
  (ct/routes [[:prefix "api"
               [:any authentication-handler]
               [:get "users" users-handler]]]))

For a better understanding of how the handler delegation chain works, see the Handlers delegation section in advanced guide chapter.

Error handling

The catacumba router chain allows to setup user defined error handling functions. This requires a very simple setup, you only have to add another route entry with using :error route directive:

(def routes
  (ct/routes [[:error my-error-handler]
              [:get "users" users-handler]]))

With the previous code we have set up a global error handler, applying to all routes in the chain. But there is also the possibility to set different error handlers for different prefixes:

(def routes
  (ct/routes [[:prefix "api"
               [:error my-error-handler-for-this-prefix]
               [:any authentication-handler]
               [:get "users" users-handler]
               [:put "users" check-permissions-handler update-users-hander]]
              [:prefix "admin"
               [:error my-error-handler-for-this-other-prefix]
               [:get "dashboard" my-dashboard-handler]]]))

The error handler signature is very similar to standard HTTP handler signature, with the difference being that it receives the throwable instance as an additional parameter:

(defn my-error-handler
  [context error]
  (http/internal-server-error (.getMessage error)))

Serving static files

Catacumba also comes with the ability to serve static files. This is can be done using :assets routing directive. Here an example:

(ct/routes [[:assets "assets" {:dir "public/assets"}]])

Additionally, it has support for specify a index file, that will be returned if no file is requested. This is very useful for SPA (single page applications):

(ct/routes [[:assets "assets" {:dir "public/assets"
                               :indexes ["index.html"]}]])

So, if you make a http request to /assets/ the index.html will be automatically returned.

Note	the assets are resolved using the `:basedir` parameter of the server constructor; for more details see the Launching the server section.

Cookies

You can access to the request cookies through direct keyword lookup on context object:

(get-in context [:cookies :somecookie])
;; => {:value "foo" :path "/" ...}

The cookies map is almost identical to the one that you can find in ring, and has the following possible properties:

:domain - restrict the cookie to a specific domain
:path - restrict the cookie to a specific path
:secure - restrict the cookie to HTTPS URLs if true
:http-only - restrict the cookie to HTTP if true (not accessible via e.g. JavaScript)
:max-age - the number of seconds until the cookie expires

For set cookies, you should use the set-cookies! function as you can see in the following example:

(ct/set-cookies! context {:cookiename {:value "foobar" :max-age 3600}})

Note	Take care that the cookie value is restricted to a very limited set of characters as spcified in RFC6265 that netty/ratpack implements. There also a relevant SO answer.

Advanced topics

Handler delegation

A part of the obvious (and previously explained) responsibility of the context object in catacubla, it has some others responsibilities. Here just a summary of them:

Here a small summary of the context responsibilities besides the obvious one explained in previous sections (IO handling):

Provide direct access to the request and response objects.
Access to the contextual objects (called registry).
Flow control in handler chaining.
Convenience helpers for common handlers operation.

In a catacumba design (inherited from ratpack), a handler is a unit of work in an asynchronous handler pipeline and the context is a execution controller and local storage for the current request state.

In other words it can be explained as "flow control" in the chain of handlers.

The request process is an asynchronous pipeline of handlers that can be composed in different ways (as we previously seen in other parts of the documentation). So the each handler in the pipeline has the ability to do some work and delegate the rest of processing to next handler in the chain.

This approach allows you build different kind of modular and completely decoupled handlers and compose them into a pipeline to work together.

The delegation response can be done with delegate function. Let see a simple example:

(defn handler1
  [context]
  (do-something context)
  (ct/delegate)

(defn handler2
  [context]
  (http/ok "hello world"))

(def router
  (ct/routes [[:get "foo" handler1 handler2]]))

In this example, when the request arrives at handler1, it delegates the execution to the next handler in the chain. It do not need to know about next handler, it just delegates to the routing chain to find a next handler or raise a corresponding error.

In addition to the simple handler delegation, catacumba offers a simple way to pass context data to the next handler in the chain. It can be done by passing an additional parameter to the delegate function:

(defn handler1
  [context]
  (do-something context)
  (ct/delegate {:message "foobar"}))

(defn handler2
  [context]
  (let [message (:message context)]
    (http/ok message)))

In the example above, the second handler prints the message found in the context.

Launching the server

Getting Started

As you can see in the quick start section, the main entry point for start the server is the run-server function that receives a handler chain and a map with options.

(require '[catacumba.core :as ct])

;; handler definition goes here

(ct/run-server #'my-handler {:port 4040 :debug true})

If you want to stop the server, you just need to call the .close method on the object returned by the run-server function.

Configuration Options

Here a complete reference of the currently supported options that can be passed to the run-server function:

Keyword Default Description

Keyword	Default	Description
`:port`	`5050`	The port used to bind the socket.
`:host`	`nil`	The host used to bind the socket.
`:threads`	(num of cores * 2)	The number of threads for handler requests.
`:debug`	`false`	Start in development mode.
`:setup`	`nil`	A callback for configuration step (low level ratpack access).
`:basedir`	`nil`	The application base directory, used mainly for resolving relative paths and assets.
`:keystore-path`	`nil`	A relative path in the classpath to the ssl keystore.
`:keystore-secret`	`nil`	A secret for the ssl key store
`:decorators`	`nil`	A vector of handlers to attach at the start of the pipeline
`:marker-file`	`.catacumba.basedir`	A file name that will be used to find the base directory in the class path.
`:max-body-size`	1048576 bytes (1mb)	A maximum length of body.

:port

5050

The port used to bind the socket.

:host

nil

The host used to bind the socket.

:threads

(num of cores * 2)

The number of threads for handler requests.

:debug

false

Start in development mode.

:setup

nil

A callback for configuration step (low level ratpack access).

:basedir

nil

The application base directory, used mainly for resolving relative paths and assets.

:keystore-path

nil

A relative path in the classpath to the ssl keystore.

:keystore-secret

nil

A secret for the ssl key store

:decorators

nil

A vector of handlers to attach at the start of the pipeline

:marker-file

.catacumba.basedir

A file name that will be used to find the base directory in the class path.

:max-body-size

1048576 bytes (1mb)

A maximum length of body.

All supported options of this function, can be overwritten at JVM startup, using environment variables or system properties. This allows to customize the server without modifying source code and exists for convenience to make easy customizations in deployments.

For example, you can change the default port on JVM startup using the CATACUMBA_PORT environment variable or catacumba.port system property:

Example using enviroment variables

export CATACUMBA_PORT=8000
export CATACUMBA_BASEDIR=`pwd`
java -jar yourjarhere.jar

Example using enviroment variables

java -Dcatacumba.port=8000 -Dcatacumba.debug=true -jar yourjarhere.jar

Note	If no `:basedir` is specified, catacumba will try to find a `.catacumba.basedir` file in the classpath and will set a base dir to its directory.

Warning

If you are deploying your application as uberjar and you want serve static files from the classpath, you should set the :marker-file to somethig different that the default value (e.g. catacumba.basedir) without the first ., because leiningen ignores all files that starts with dot.

SSL Configuration

Catacumba server can be configured to use TLS (commonly known as SSL). The process is pretty simple but it requires to have a proper key and certificate.

The first thing that you should care about is that catacumba is built on jvm that the default ssl certificate/key format used by nginx/apache it isn’t compatible but is very easy create a compatible file using the openssl command line.

Having a key and the certificate, just execute this command:

openssl pkcs12 -export -in cert.pem -inkey key.pem -out store.p12

This process will ask you for a password that you must memorize and later provide it to catacumba. Now, having the properly formated trusted store, just pass some additional parameters on starting the server:

(ct/run-server #'my-handler {:port 4040
                             :keystore-secret "yoursecrethere"
                             :keystore-path "path/to/store.p12"})

Note	catacumba at this moment does not has the "upgrade" approach so if you setup ssl, only ssl connections will be accepted. So the most recommended way to use ssl on your application is put catacumba behind nginx or haproxy and make them handle the ssl.

Handler types

This section intends to explain the different kind of built in handler types and the response types that comes out of the box with catacumba. This section is organized on handler types as first level and possible supported return values as second level.

Asynchronous

Asynchronous handlers are handlers that return a value in an asynchronous way using one of the supported abstractions, such as core.async, reactive-streams and many others (explained below).

Channel (core.async)

The core.async channel is one of the supported abstractions that comes with catacumba out of the box. It consists of a handler that returns a body as a channel or response as a channel.

This is the aspect of async handler returning a core.async channel as a body:

(defn my-async-handler
  [context]
  (let [ch (chan)]
    (go
      (dotimes [i 10]
        (<! (timeout 500))
        (>! ch (str i "\n")))
      (close! ch))
    (http/ok ch)))

Do not worry about how much data you can send to the client, if you are using channels in a right way (in a go block), you will send data to the client as fast as the client can consume it. This technique is also called back pressure, and is fully supported for chunked responses.

Additionally, you also can return a channel as the handler response. The main difference is that in this case you should put a complete response into the channel:

(defn my-async-handler
  [context]
  (go
    (let [result (<! (do-some-async-task))]
      (http/ok (:data result)))))

CompletableFuture

Sometimes, you do not need send a chunked stream to the client, but your "business logic" is defined in an asynchronous friendly API using promises (or something similar). In this case, with catacumba you can return a promise as a body or as a response and the data will be sent to the client when the promise has been resolved successfully.

The CompletableFuture is an other asynchronous primitive supported out of the box by catacumba; so you can return it as body or as response out of the box.

For more pleasant usage of CompletableFuture in clojure, the promesa library is used. That library provides a more clojure friendly api on top of JDK8 CompletableFuture and a great sugar syntax for composing them.

A example using the promesa library api for create a CompletableFuture instance and return it as body.

(require '[promesa.core :as p])

(defn my-async-handler
  [context]
  (let [prm (p/resolved "hello world")]
    (http/ok prm {:content-type "text/plain"})))

Like as usual, you can return an instance of CompletableFuture as response:

(require '[promesa.core :as p])

(defn my-async-handler
  [context]
  (p/promise (fn [resolve]
               (future
                 (Thread/sleep 100)
                 (resolve (http/ok "hello world"))))))

One of the advantages of using CompletableFuture abstraction with promesa library is because it exposes additional sugar syntax that help workin with asynchronous flows in more pleasant way.

Let see an example:

(require '[promesa.core :as p])

(defn my-async-handler
  [context]
  (p/alet [a (p/await (some-async-op1))
           b (p/await (some-async-op@))
           result (str a b)]
    (http/ok result)))

The result of alet macro expression will be an instance of CompletableFuture that eventually will be completed with the http response.

Manifold Deferred

The manifold library also offers a promise like abstraction. The main advantage of using it is that is build for clojure and is not restricted to JDK8.

Example code that returns a body as manifold deferred.

(require '[manifold.deferred :as d])

(defn my-async-handler
  [context]
  (let [result (d/future
                 (Thread/sleep 1000)
                 "hello world")]
    (http/ok result {"content-type" "text/plain"})))

Like the previously explained abstractions, you also can return manifold deferreds as handler response.

Reactive-Streams

The reactive-streams support is inherited from ratpack and like manifold streams it is only can be used for send the response body.

Here there isn’t anything new to explain, just build and/or compose your streams and return them as http response body:

(require '[catacumba.stream :as stream])
(require '[cuerdas.core :as str])

(defn my-async-handler
  [context]
  (let [pub (->> (stream/publisher ["hello" " " "world"])
                 (stream/transform (map str/upper)))]
    (http/ok pub)))

;; It will return a chunked response to the client with "HELLO WORLD" string.

One of the best parts of the reactive-strams is that them comes with back pressure support out of the box and it native support in ratpack makes them a great glue abstraction for similar async primitives. In fact, the support for all stream like primitives explained until now are implemented in terms of reactive-streams publisher.

WebSockets

One of the main goals of catacumba is come with builtin, full featured and back pressure aware websockets support.

You can start a websocket connection in any catacumba handler or route handler using websocket function. It does not require any special handlers for dealing with websockets. Let see an example:

(defn my-websocket-echo-handler
  [{:keys [in out]}]
  (go-loop []
    (if-let [received (<! in)]
      (do
        (>! out received)
        (recur))
      (close! out))))

(defn my-handler
  [context]
  (ct/websocket context my-websocket-echo-handler))

(def route
  (ct/routes [[:prefix "events"
               [:any my-handler]]]))

Additionally, catacumba offers a way to set up a websocket handler directly, without an additional step:

(defn echo-handler
  "This is my echo handler that serves as
  a websocket handler example."
  {:handler-type :catacumba/websocket}
  [{:keys [in out]}]
  (go-loop []
    (if-let [received (<! in)]
      (do
        (>! out received)
        (recur))
      (close! out))))

(def route
  (ct/routes [[:prefix "events"
               [:any #'echo-handler]]]))

As you can observe, the var metadata is used for properly choice the right adapter.

Note	Is very important pass a var reference to the router instead of the function directly, because the metadata defined in the function is bound to the var and not to the function.

Also, you can attach metadata inline, using the with-meta Clojure built-in function:

(ct/routes [[:prefix "events"
             [:any (with-meta echo-handler
                     {:handler-type :catacumba/websocket})]]])

Clojure offers a lot of flexibility for working with metadata so you can set the handler type in the way that you prefer.

SSE (Server-Sent Events)

WebSockets are cool because they allow bi-directional communication, but in some circumstances we only need something unidirectional, for notifying the client about some changes or any other events. For this purpose exists Server-Sent Events (SSE) and catacumba also has support for it.

The handler for SSE does not differs much from websockets (that we have seen in the previous section). The main difference is that server-sent events are unidirectional and they only can send data in the server to client direction.

(defn time-notification
  "Handler that notifies each second
  the current server time to the client."
  {:handler-type :catacumba/sse}
  [{:keys [out ctrl] :as context}]
  (go-loop []
    (when-let [_ (>! out (str (java.time.Instant/now)))]
      (<! (timeout 1000))
      (recur))))

(def route
  (ct/routes [[:prefix "events"
               [:any #'time-notification]]]))

In a similar way to websockets, you can start SSE in any place, such as a standard catacumba handler:

(defn time-notification
  "Handler that notifies each second
  the current server time to the client."
  [context]
  (ct/sse context
          (fn [{:keys [out ctrl]}]
            (go-loop []
              (when-let [_ (>! out (str (java.time.Instant/now)))]
                (<! (timeout 1000))
                (recur))))))

(def route
  (ct/routes [[:prefix "events"
               [:any time-notification]]]))

Let see some examples how you can send other parameters than simple data:

;; Send data
(>! out "data as string")
(>! out {:data "data as string"})

;; Send data with event name
(>! out {:data "data as string" :event "foobar"})

;; Set id
(>! out {:id "2"})

Note

The catacumba's SSE support uses core.async channels, but if you are not happy with core.async and want use something different (such as manifold streams or beicon), you may want know that everything in catacumba is implemented using abstractions and to implement your own SSE type of handler that uses manifold streams is very easy.

CPS (Continuation-passing style)

Is a low level handler type that works in a cps style (in other words, they works with callbacks). This is not general purpose handler type but you maybe found it useful for integrate catacumba with other scenarios that it is not initially designed to work.

This is the aspect ot the cps style handler:

(defn my-cps-handler
  "Some usefull docstring."
  {:handler-type :catacumba/cps}
  [context callback]
  (future
    (Thread/sleep 1000)
    (callback "hello world")))

Built-in Handlers

This section will cover different kind of built-in additional handlers to make the experience of using catacumba more pleasant.

Body parsing

Catacumba comes with builtin support for conditional body parsing depending on the incoming content type. It consists of a routing chain handler that adds the :data entry in the context with the parsed data or nil in case of an incoming content type does not have an attached parser implementation.

In order to use it you should prepending the body-params handler to your route chain:

(require '[catacumba.handlers.parse :as parse])

(defn example-handler
  [context]
  (let [body (:data context)]
    (println "Received data:" body)
    (http/no-content)))

(def app
  (ct/routes [[:any (parse/body-params)]
              [:any example-handler]]))

;; ...

By default, the application/x-www-form-urlencoded, multipart/form-data, application/json, application/transit+json and application/transit+msgpack parsers come out of the box. The cheshire json parser is used for parsing the body with the application/json content type.

The body parsing is a open system, implemented using clojure’s polymorphism facilities such as multimethods. If you want add additional parser, just add an additional implementation to the parse multimethod with your content-type as dispatch tag.

(require '[catacumba.handlers.parse :as parse])
(import 'ratpack.http.TypedData
        'ratpack.handling.Context)

(defmethod parse/parse-body :application/xml
  [^Context ctx ^TypedData body]
  ;; your parsing logic here
  )

Autoreload

The autoreload handler consist in a very simple concept: reload all modified namespaces on each request. If you are familiar with the ring reload middleware, this one works in almost identical way.

For use it, just attach it to your routing chain:

(require '[catacumba.handlers.misc :as misc])

(def app
  (ct/routes [[:any (misc/autoreloader)]
              [:get "foo" #'somens/your-handler]
              [:get "bar" #'somens/other-handler]
              [:post ...]]))

You can see a working example in the Website example code.

Tip	The auto-reloading can only work if you pass var references to the router definition instead resolved values. In same way as the previous example.

Sessions

Getting Started

The HTTP sessions in catacumba are also implemented as chain handler. So you can add session handling support to you application just by adding the handler to your routing chain:

(require '[catacumba.handlers.session :as session])

(def app
  (ct/routes [[:any (session/session {:storage :inmemory})]
              [:get your-handler]]))

All handlers in the route pipeline that are going after the session handler will come with :session key in the context with a "atom" like object. You just treat it as atom, so for attaching some data to the session you should use the well known swap! function:

(defn my-handler
  [context]
  (let [session (:session context)]
    (swap! session assoc :userid 1)
    "my response"))

You can clean the session just reseting to the empty map:

(reset! session {})

One of the big advantages of using the routing chain for session set up, is that you can restrict session handling to a concrete subset of urls/resources avoiding unnecessary code execution for handlers that do not need sessions:

(def app
  (ct/routes [[:prefix "admin"
               [:any (session/session {:storage :inmemory})]
               [:get your-handler]]
              [:prefix "api"
               [:get "users" other-handler]
               [:get ...]]]))

Session storages

Currently catacumba comes with one basic session storage, the :inmemory. But the session storage system is pluggable and is defined in terms of the following protocol:

(defprotocol ISessionStorage
  (read-session [_ key])
  (write-session [_ key data])
  (delete-session [_ key]))

If you are familiar with the ring based session storages, you can observe that the catacumba session storage abstraction is almost identical to the ring session abstraction, so migrating from or adapting the ring session storages is really easy. The unique difference is that functions should return a promise (from promesa library).

To use a concrete session storage, just pass a instance of it as value of the :storage key in a session handler constructor:

(session/session {:storage (my-storage-constructor)})

If you want implement own session storage, take a look to the :inmemory builtin one.

Authentication

Catacumba also comes with authentication facilities heavily inspired by buddy-auth.

We do not have used directly buddy-auth because it is designed for ring based applications, therefore the buddy-auth abstractions are blocking, and blocking api is not well suited for async based applications.

So, catacumba defines own abstractions for handle authentication, that are very very similar to the buddy-auth, with the exception that them expose asynchronous api, so adapt existing buddy-auth backends should be very easy.

Like buddy-auth, catacumba comes with a little set of builtin backends that can be used directly: session, jws (token) and jwe (encrypted token).

Session

Let start with session authentication backend. This backend is mainly used for web based applications and consists in verify some value on the session. So this is the easiest authentication scheme and fits perfectly for the first contact.

Start importing some needed namespaces and create an instance of the authentication backend:

(require '[catacumba.http :as http])
(require '[catacumba.handlers.auth :as cauth])

(def auth-backend
  (cauth/session-backend))

Now, continue defining a handler for the login action. It consists in receive credentials from the user input and verify them. In case of success verification, we just need setup the :identity key in the session.

Let see a partially implemented example:

(defn login-handler
  [context]
  (let [data (:body context)
        user (find-user (:username data)   ;; (implementation omitted)
                        (:password data))]
    (swap! (:session context) assoc :identity user)
    (http/ok "ok")))

In order to start using auth facilities in your application, you should add the authentication handler to the routing chain:

;; The application routes definition with session, auth and body
;; parsing chain handlers

(def app
  (ct/routes [[:any (session/session {:storage :inmemory})] ;; Http Session
              [:any (cauth/auth auth-backend)]              ;; Auth backend
              [:any (parse/body-params)]                  ;; Body parsing
              [:get "login" login-handler]
              [:get some-handler]]))                     ;; (implementation omitted)

You can see a working example using auth facilities here.

JWS Token

This authentication backend consists in use self contained tokens for authenticate the user. It behaves very similar to the session one but instead of strong the user information in a server storage, it stores it directly in a token, enabling so, completely stateless authentication.

Note	The security and the implementation of cryptographic primitives for that token is relied to the buddy-sign library (an other module of buddy) that implements the JWS specification. That library should be used for generate JWS tokens.

Let start creating a backend instance:

(def secret "mysecret")
(def auth-backend
  (cauth/jws-backend {:secret secret}))

Following of our new login handler:

(require '[buddy.sign.jwt :as jwt])
(require '[cheshire.core :as json])

(defn login-handler
  [context]
  (let [data (:body context)
        user (find-user (:username data)   ;; (implementation omitted)
                        (:password data))]
    (-> (json/encode {:token (jwt/sign {:user (:id user)} secret)})
        (http/ok {:content-type "application/json"}))))

And finally, put the new backend into the routing chain:

(def app
  (ct/routes [[:any (cauth/auth auth-backend)]     ;; Auth backend
              [:any (parse/body-params)]           ;; Body parsing
              [:get "login" login-handler]
              [:get some-handler]]))               ;; (implementation omitted)

Warning

Take care that using jws for create tokens, the data is serialized using json + base64 and signed using strong cryptography signatures. That method ensure that the data can not be manipulated by third party but it not protect it from privacy. If you need store private data in the token, consider using JWE.

JWE Token

This authentication backend consists in using self contained tokens for authenticate the user. It works identically to the JWS (explained previously) with the exception that instead of only signing data, it also encrypts the data, so ensuring the data privacy.

You can create the backend instance so:

(require '[buddy.sign.jwt :as jwt])
(require '[buddy.core.keys :as keys])

(def pubkey (keys/public-key "pubkey.pem"))
(def privkey (keys/private-key "privkey.pem" "thekeysecret"))

(def auth-backend
  (auth/jwe-backend privkey))

Note	In this example we use asymmetric encryption scheme, if you want use an other encryption scheme, please check buddy-sign documentation for the complete list of supported encryption algorithms.

The login handler is almost identical:

(require '[buddy.sign.jwt :as jwt])
(require '[cheshire.core :as json])

(defn login-handler
  [context]
  (let [data (:body context)
        user (find-user (:username data)   ;; (implementation ommited)
                        (:password data))]
    (-> (json/encode {:token (jwt/encrypt {:user (:id user)} pubkey)})
        (http/ok {:content-type "application/json"}))))

Instead of signing the content, we encrypt it using the public key. The routing chain is completely identical from the JWE Token examples.

Other

If you not happy with the builtin auth facilities, the catacumba's handler system is very flexible and you really don’t need to use buddy. You can write your own auth facilities and attach them to catacumba using the routing chain.

Security

Cross-Origin Resource Sharing (CORS) is a mechanism that allows restricted resources (e.g. fonts, JavaScript, etc.) on a web page to be requested from another domain outside the domain from which the resource originated.

Is often used for allowing API resources to be accessed in a web browser, out of the domain of your web applications.

Catacumba has builtin support for CORS, and this is how you can use it:

(require '[catacumba.handlers.misc :as misc])

(def cors-conf {:origin #{"http://website.com"}                     ;; mandatory
                :max-age 3600                                       ;; optional
                :allow-methods #{:post :put :get :delete}           ;; optional
                :allow-headers #{:x-requested-with :content-type}}) ;; optional

(def app
  (ct/routes [[:prefix "api/v1"
               [:any (misc/cors cors-conf)]
               [:get "foo" some-handler]
               [:post "foo" some-save-handler]]]))

The :origin key can be a set of possible origins or simply "*" to allow all origins.

Content Security Policy

Is a security related chain handler that appropriately sets the Content-Security-Policy headers.

Content Security Policy (CSP) is an added layer of security that helps to detect and mitigate certain types of attacks, including Cross Site Scripting (XSS) and data injection attacks. These attacks are used for everything from data theft to site defacement or distribution of malware.

Here a simple example on how to use it:

(def cspconf {:default-src "'self' *.trusted.com"
              :img-src "*"
              :frame-ancestors "'none'"
              :reflected-xss "filter"})

(def app
  (ct/routes [[:prefix "web"
               [:any (csp-headers cspconf)]
               [:get your-handler]]])

You can read more about that here: https://developer.mozilla.org/en-US/docs/Web/Security/CSP. The complete list of directives can be found here: https://developer.mozilla.org/en-US/docs/Web/Security/CSP/CSP_policy_directives

This handler supports the following directives: :default-src, :frame-ancestors, :frame-src, :child-src, :connect-src, :font-src, :form-action, :img-src, :media-src, :object-src, and :reflected-xss.

Frame Options

This is a security related chain handler that adds X-Frame-Options header to the response.

The X-Frame-Options HTTP response header can be used to indicate whether or not a browser should be allowed to render a page in a <frame>, <iframe> or <object> . Sites can use this to avoid clickjacking attacks, by ensuring that their content is not embedded into other sites.

Example:

(require '[catacumba.handlers.security :as sec])

(def app
  (ct/routes [[:prefix "web"
               [:any (sec/frame-options-headers {:policy :deny})]
               [:get your-handler]]]))

The possible values for the :policy key are: :deny and :sameorigin.

Warning

The frame-ancestors directive from the CSP Level 2 specification officially replaces this non-standard header.

Strict Transport Security

This is a security related chain handler that adds the Strict-Transport-Security header to the response.

HTTP Strict Transport Security (often abbreviated as HSTS) is a security feature that lets a web site tell browsers that it should only be communicated with using HTTPS, instead of using HTTP.

Usage example:

(require '[catacumba.handlers.security :as sec])

(def app
  (ct/routes [[:prefix "web"
               [:any (sec/hsts-headers {:max-age 31536000 :subdomains true })]
               [:get your-handler]]]))

You can read more about that header here: https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security

Content Type Options

This is a security related chain handler that adds the X-Content-Type-Options header to the response. It prevents resources with invalid media types being loaded as stylesheets or scripts.

This chain handler does not have any additional parameters. Let see an example on how you can use it:

(require '[catacumba.handlers.security :as sec])

(def app
  (ct/routes [[:prefix "web"
               [:any sec/content-type-options-headers]
               [:get your-handler]]]))

More information:

CSRF (Cross-Site Request Forgery)

This is a security related chain handler that protects the following handlers from one-click attack.

For use it, just add it to your routing pipeline:

(require '[catacumba.handlers.security :as sec])

(def app
  (ct/routes [[:prefix "web"
               [:any (sec/csrf-protect)]
               [:get your-handler]]]))

The response will be populated automatically with csrftoken cookie that should be read by the client side javascript and put the same value under the x-csrftoken header or under csrftoken form encoded field.

If you want access to the current value of the csrftoken inside catacumba handler, you can do it using :catacumba.handlers.security keyword lookup on the context.

More information:

Request logging

catacumba by default does not logs almost anything in console, and the request logging is not an exception. This is a good default and is very recommended use reverse proxy logging facilities.

But, if you want request logging in catacumba, you can easy activate it just attaching additional handler to your routing chain:

(require '[catacumba.handlers.misc :as misc])

(def app
  (ct/routes [[:any (misc/log)]
              ;; here your handlers
              ]))

The default implementation in most cases is more than enough, but if you don’t happy with it you can provide your own function for logging:

(defn my-logging-handler
  [context, outcome]
  (println context outcome))

(def app
  (ct/routes [[:any (misc/log my-logging-handler)]
              ;; here your handlers
              ]))

The context parameter is just a context that you have already used previously, and outcome is hash map that contains additional data such as: response headers, respose status and request duration time.

Instrumentation

TODO

Plugins

This section will explain useful modules that are not part of the core of catacumba but are fully supported.

catacumba-prone

Prone is a exception reporting middleware for ring based applications that show a beautiful, navigable and human readable stacktraces when an exception is throwed in your application.

Full documentation: https://github.com/funcool/catacumba-prone

Note	You can found a complete example using prone in the examples section.

Examples

Website and Auth

This example tries to show the way to use catacumba in a website like projects, with authentication and sessions.

Just run the following commands:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/
$ lein with-profile website-example run
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

You can found the source code of this example here.

WebSocket Echo

This example application tries to show a very simple application using the websockets capabilities of catacumba

Get it up and running following this commands:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/
$ lein with-profile websocket-echo-example run
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

You can found the source code of this example here.

Multiuser chat with SSE

This example tries to demonstrate how can you build a simple chat using "Server-Sent Events" for communicating events to the client:

For make this example application run, follow this commands:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/
$ lein with-profile sse-chat-example run
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

Now, open http://localhost:5050 in two different browsers and try send messages between them.

You can found the source code of this example here.

Debugging with prone

Prone is really awesome middleware for ring that shows a beautiful and human readable stack traces when a exception is raised in your application.

Just follow the following commands for get it up and running:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/
$ lein with-profile debugging-example run
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

You can found the source code of this example here.

Note	Obviously, if you are using the ring type of handler, you can use Prone as is, without any additional adaptation. This example shows how it can be used with catacumba's default handler type.

Instrumentation

Catacumba comes with the ability to instrument your application for taking different kinds of diagnosis, such as performance, latency, etc. This example shows how it can be done.

In case of this concrete example application, it uses the instrumentation facilities of catacumba for monitoring the time of execution of request handlers.

Follow this steps for get this example up and running:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/
$ lein with-profile interceptor-example run
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

And go to http://localhost:5050/

After some requests, you will see the similar output in the console:

Computation :compute elapsed in: 0.025150461 (sec)
Computation :compute elapsed in: 0.001690894 (sec)
Computation :compute elapsed in: 0.001541675 (sec)
Computation :compute elapsed in: 0.001554894 (sec)
Computation :compute elapsed in: 0.00175033 (sec)

You can found the source code of this example here.

Single file web app

This example application requires that you should have boot-clj properly installed on your system.

This example tries to show how you can use catacumba for building small web applications that fits in one file and execute them like a shell script or an executable.

You should execute the following commands for get it up and running:

$ git clone git@github.com:funcool/catacumba.git
$ cd catacumba/examples/single-file
$ export BOOT_CLOJURE_VERSION=1.7.0
$ ./main.clj
[main] INFO ratpack.server.RatpackServer - Ratpack started for http://localhost:5050

You can found the source code of this example here.

FAQ

What is the difference between catacumba and Aleph?

First of all, Aleph is not a real alternative to catacumba, because its approach is so much low level and its web server support is a little bit constrained by ring spec. Furthermore, aleph is already used in catacumba as http client in tests code and manifold (async abstractions behind aleph) is a first class abstractions for handle async values.

So, I’m happy to tell you that you can use the both libraries together because they are very complementary.

What is the rationale behind this project?

I started writing this library as a research project to provide a simple, non obstructive (a la ring) without the constraints of the existing ring spec. The aim is to create a web toolkit for building asynchronous web services.

Here is an incomplete list of things that catacumba aims to achieve:

Allow different types of handlers by being flexible and extensible
Provide a simple and lightweight approach for defining asynchronous web services with support for different abstractions such as promises, futures, core.async, manifold, reactive-streams, etc…
Build upon abstractions with simplicity and extensibility in mind.
Provide built in declarative style routing.
Remain unopinionated and versatile.
Come with back pressure support out of the box.

catacumba is not designed:

To be a fully integrated full stack solution like Immutant or Pedestal.
To provide an opinionated way to structure your "business logic"
To provide all possible features that you might need.
To be a low level, ring based library.

The result of this research project is a powerful, lightweight, and fully extensible asynchronous web toolkit built on top of existing and well designed components such as Ratpack and Netty.

Developers Guide

Philosophy

Five most important rules:

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Readability counts.

All contributions to catacumba should keep these important rules in mind.

Contributing

Unlike Clojure and other Clojure contributed libraries catacumba does not have many restrictions for contributions. Just open an issue or pull request.

Source Code

catacumba is open source and can be found on github.

You can clone the public repository with this command:

git clone https://github.com/funcool/catacumba

Run tests

For running tests just execute this:

lein test

License

catacumba is licensed under BSD (2-Clause) license:

Copyright (c) 2015-2016 Andrey Antukh <niwi@niwi.nz>

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.