1. About this book

This book covers the ClojureScript programming language, serves as a detailed guide of its tooling for development, and presents a series of articles about topics that are applicable to day-to-day programming in ClojureScript.

It is not an introductory book to programming in that it assumes the reader has experience programming in at least one language. However, it doesn’t assume experience with Clojure or functional programming. We’ll try to include links to reference material when talking about the theoretical underpinnings of ClojureScript that may not be familiar to everyone.

Since the ClojureScript documentation is good but sparse, we wanted to write a compendium of reference information and extensive examples to serve as a ClojureScript primer as well as a series of practical how-to’s. This document will evolve with the ClojureScript language, both as a reference of the language features and as a sort of cookbook with practical programming recipes.

You’ll get the most out of this book if you:

  • are curious about ClojureScript or functional programming and have some programming experience;

  • write JavaScript or any other language that compiles to it and want to know what ClojureScript has to offer;

  • already know some Clojure and want to learn how ClojureScript differs from it, plus practical topics like how to target both languages with the same code base.

Don’t be turned off if you don’t see yourself in any of the above groups. We encourage you to give this book a try and to give us feedback on how we can make it more accessible. Our goal is to make ClojureScript more friendly to newcomers and spread the ideas about programming that Clojure has helped popularize, as we see a lot of value in them.

This is a list of translations of the book:

2. Introduction

Why are we doing this? Because Clojure rocks, and JavaScript reaches.
— Rich Hickey

ClojureScript is an implementation of the Clojure programming language that targets JavaScript. Because of this, it can run in many different execution environments including web browsers, Node.js, Nashorn (and many other).

Unlike other languages that intend to compile to JavaScript (like TypeScript, FunScript, or CoffeeScript), ClojureScript is designed to use JavaScript like bytecode. It embraces functional programming and has very safe and consistent defaults. Its semantics differ significantly from those of JavaScript.

Another big difference (and in our opinion an advantage) over other languages is that Clojure(Script) is designed to be a guest. It is a language without its own virtual machine that can be easily adapted to the nuances of its execution environment. This has the benefit that Clojure (and hence ClojureScript) has access to all the existing libraries written for the host language.

Before we jump in, let us summarize some of the core ideas that ClojureScript brings to the table. Don’t worry if you don’t understand all of them right now, they’ll become clear throughout the book.

  • ClojureScript enforces the functional programming paradigm with its design decisions and idioms. Although being strongly opinionated about functional programming it’s a pragmatic language rather than pursuing theoretical purity.

  • Encourages programming with immutable data, offering highly performant and state of the art immutable collection implementations.

  • It makes a clear distinction of identity and its state, with explicit constructs for managing change as a series of immutable values over time.

  • It has type-based and value-based polymorphism, elegantly solving the expression problem.

  • It is a Lisp dialect so programs are written in the programming language’s own data structures, a property known as homoiconicity that makes metaprogramming (programs that write programs) as simple as it can be.

These ideas together have a great influence in the way you design and implement software, even if you are not using ClojureScript. Functional programming, decoupling of data (which is immutable) from the operations to transform it, explicit idioms for managing change over time and polymorphic constructs for programming to abstractions greatly simplify the systems we write.

We can make the same exact software we are making today with dramatically simpler stuff — dramatically simpler languages, tools, techniques, approaches.
— Rich Hickey

We hope you enjoy the book and ClojureScript brings the same joy and inspiration that has brought to us.

3. Language (the basics)

This chapter will be a little introduction to ClojureScript without assumptions about previous knowledge of the Clojure language, providing a quick tour over all the things you will need to know about ClojureScript and understand the rest of this book.

You can run the code snippets in the online interactive repl: http://www.clojurescript.io/

3.1. First steps with Lisp syntax

Invented by John McCarthy in 1958, Lisp is one of the oldest programming languages that is still around. It has evolved into many derivatives called dialects, ClojureScript being one of them. It is a programming language written in its own data structures — originally lists enclosed in parentheses — but Clojure(Script) has evolved the Lisp syntax with more data structures, making it more pleasant to write and read.

A list with a function in the first position is used for calling a function in ClojureScript. In the example below, we apply the addition function to three arguments. Note that unlike in other languages, + is not an operator but a function. Lisp has no operators; it only has functions.

(+ 1 2 3)
;; => 6

In the example above, we’re applying the addition function + to the arguments 1, 2 and 3. ClojureScript allows many unusual characters like ? or - in symbol names, which makes it easier to read:

(zero? 0)
;; => true

To distinguish function calls from lists of data items, we can quote lists to keep them from being evaluated. The quoted lists will be treated as data instead of as a function call:

'(+ 1 2 3)
;; => (+ 1 2 3)

ClojureScript uses more than lists for its syntax. The full details will be covered later, but here is an example of the usage of a vector (enclosed in brackets) for defining local bindings:

(let [x 1
      y 2
      z 3]
  (+ x y z))
;; => 6

This is practically all the syntax we need to know for using not only ClojureScript, but any Lisp. Being written in its own data structures (often referred to as homoiconicity) is a great property since the syntax is uniform and simple; also, code generation via macros is easier than in any other language, giving us plenty of power to extend the language to suit our needs.

3.2. The base data types

The ClojureScript language has a rich set of data types like most programming languages. It provides scalar data types that will be very familiar to you, such as numbers, strings, and floats. Beyond these, it also provides a great number of others that might be less familiar, such as symbols, keywords, regexes (regular expressions), vars, atoms, and volatiles.

ClojureScript embraces the host language, and where possible, uses the host’s provided types. For example: numbers and strings are used as is and behave in the same way as in JavaScript.

3.2.1. Numbers

In ClojureScript, numbers include both integers and floating points. Keeping in mind that ClojureScript is a guest language that compiles to JavaScript, integers are actually JavaScript’s native floating points under the hood.

As in any other language, numbers in ClojureScript are represented in the following ways:

23
+23
-100
1.7
-2
33e8
12e-14
3.2e-4
##Inf
##-Inf

3.2.2. Keywords

Keywords in ClojureScript are objects that always evaluate to themselves. They are usually used in map data structures to efficiently represent the keys.

:foobar
:2
:?

As you can see, the keywords are all prefixed with :, but this character is only part of the literal syntax and is not part of the name of the object.

You can also create a keyword by calling the keyword function. Don’t worry if you don’t understand or are unclear about anything in the following example; functions are discussed in a later section.

(keyword "foo")
;; => :foo

In the end, keywords are a special case of Symbols which will be explained later.

Namespaced keywords

When prefixing keywords with a double colon ::, the keyword will be prepended by the name of the current namespace. Note that namespacing keywords affects equality comparisons.

::foo
;; => :cljs.user/foo

(= ::foo :foo)
;; => false

Another alternative is to include the namespace in the keyword literal, this is useful when creating namespaced keywords for other namespaces:

:cljs.unraveled/foo
;; => :cljs.unraveled/foo

The keyword function has an arity-2 variant where we can specify the namespace as the first parameter:

(keyword "cljs.unraveled" "foo")
;; => :cljs.unraveled/foo

Just as keywords are usually used to define keys on a map, namespaced keywords allow defining attributes dedicated to a specific domain without colliding with other keywords that may have the same name.

3.2.3. Symbols

Symbols in ClojureScript are very, very similar to keywords (which you now know about). But instead of evaluating to themselves, symbols are evaluated to something that they refer to, which can be functions, variables, etc.

Symbols start with a non numeric character and can contain alphanumeric characters as well as *, +, !, -, _, ', and ? such as :

sample-symbol
othersymbol
f1
my-special-swap!

The symbols themselves are not usually used as data but are used by the language to resolve function calls. In other words, let’s use this example:

(println hello-world)

In this case, the println symbol is telling the compiler to look in its resolution table to see if it has a bound implementation of a function that prints data to standard output.

And by wrapping the symbol in parentheses, you are telling the compiler that what you want is to execute that function. All elements other than the println symbol, which is in the first place, are considered parameters.

3.2.4. Strings

There is almost nothing new we can explain about strings that you don’t already know. In ClojureScript, they work the same as in any other language. One point of interest, however, is that they are immutable.

In this case they are the same as in JavaScript:

"An example of a string"

One peculiar aspect of strings in ClojureScript is due to the language’s Lisp syntax: single and multiline strings have the same syntax:

"This is a multiline
      string in ClojureScript."

3.2.5. Characters

ClojureScript also lets you write single characters using Clojure’s character literal syntax.

\a        ; The lowercase a character
\newline  ; The newline character

Since the host language doesn’t contain character literals, ClojureScript characters are transformed behind the scenes into single character JavaScript strings.

3.2.6. Booleans

The same for booleans:

true
false

The really interesting part of the booleans is the concepts of logical booleans. Is a concept in which we assign a boolean meaning to an expression (if it is falsy or truthy even if it is not of boolean type).

This is the aspect where each language has its own semantics (mostly wrongly in my opinion). The majority of languages consider empty collections, the integer 0, and other things like this to be false. In ClojureScript, unlike in other languages, only two values are considered as logical false: nil and false. Everything else is treated as logical true.

(boolean nil)     ;; => false
(boolean 0)       ;; => true
(boolean [])      ;; => true
(boolean :false)  ;; => true

3.2.7. Collections

Another big step in explaining a language is to explain its collections and collection abstractions. ClojureScript is not an exception to this rule.

ClojureScript comes with many types of collections. The main difference between ClojureScript collections and collections in other languages is that they are persistent and immutable.

Before moving on to these (possibly) unknown concepts, we’ll present a high-level overview of existing collection types in ClojureScript.

Lists

This is a classic collection type in languages based on Lisp. Lists are the simplest type of collection in ClojureScript. Lists can contain items of any type, including other collections.

Lists in ClojureScript are represented by items enclosed between parentheses:

'(1 2 3 4 5)
'(:foo :bar 2)

As you can see, all list examples are prefixed with the ' char. This is because lists in Lisp-like languages are often used to express things like function or macro calls (just as we have seen before). In that case, the first item should be a symbol that will evaluate to something callable, and the rest of the list elements will be function arguments. However, in the preceding examples, we don’t want the first item as a symbol; we just want a list of items.

The following example shows the difference between a list without and with the preceding single quote mark:

(inc 1)
;; => 2

'(inc 1)
;; => (inc 1)

As you can see, if you evaluate (inc 1) without prefixing it with ', it will resolve the inc symbol to the inc function and will execute it with 1 as the first argument, returning the value 2.

You can also explicitly create a list with the list function:

(list 1 2 3 4 5)
;; => (1 2 3 4 5)

(list :foo :bar 2)
;; => (:foo :bar 2)

Lists have the peculiarity that they are very efficient if you access them sequentially or access their first elements, but a list is not a very good option if you need random (index) access to its elements.

Vectors

Like lists, vectors store a series of values, but in this case, with very efficient index access to their elements, as opposed to lists, which are evaluated in order. Don’t worry; in the following sections we’ll go in depth with details, but at this moment, this simple explanation is more than enough.

Vectors use square brackets for the literal syntax; let’s see some examples:

[:foo :bar]
[3 4 5 nil]

Like lists, vectors can contain objects of any type, as you can observe in the preceding example.

You can also explicitly create a vector with the vector function, but this is not commonly used in ClojureScript programs:

(vector 1 2 3)
;; => [1 2 3]

(vector "blah" 3.5 nil)
;; => ["blah" 3.5 nil]
Maps

Maps are a collection abstraction that allow you to store key/value pairs. In other languages, this type of structure is commonly known as a hash-map or dict (dictionary). Map literals in ClojureScript are written with the pairs between curly braces.

{:foo "bar", :baz 2}
{:alphabet [:a :b :c]}
Note
Commas are frequently used to separate a key-value pair, but they are completely optional. In ClojureScript syntax, commas are treated like spaces.

Like vectors, every item in a map literal is evaluated before the result is stored in a map, but the order of evaluation is not guaranteed.

Sets

And finally, sets.

Sets store zero or more unique items of any type and are unordered. Like maps, they use curly braces for their literal syntax, with the difference being that they use a # as the leading character. You can also use the set function to convert a collection to a set:

#{1 2 3 :foo :bar}
;; => #{1 :bar 3 :foo 2}
(set [1 2 1 3 1 4 1 5])
;; => #{1 2 3 4 5}

In subsequent sections, we’ll go in depth about sets and the other collection types you’ve seen in this section.

3.3. Vars

ClojureScript is a mostly functional language that focuses on immutability. Because of that, it does not have the concept of variables as you know them in most other programming languages. The closest analogy to variables are the variables you define in algebra; when you say x = 6 in mathematics, you are saying that you want the symbol x to stand for the number six.

In ClojureScript, vars are represented by symbols and store a single value together with metadata.

You can define a var using the def special form:

(def x 22)
(def y [1 2 3])

Vars are always top level in the namespace (which we will explain later). If you use def in a function call, the var will be defined at the namespace level, but we do not recommend this - instead, you should use let to define variables within a function.

3.4. Functions

3.4.1. The first contact

It’s time to make things happen. ClojureScript has what are known as first class functions. They behave like any other type; you can pass them as parameters and you can return them as values, always respecting the lexical scope. ClojureScript also has some features of dynamic scoping, but this will be discussed in another section.

If you want to know more about scopes, this Wikipedia article is very extensive and explains different types of scoping.

As ClojureScript is a Lisp dialect, it uses the prefix notation for calling a function:

(inc 1)
;; => 2

In the example above, inc is a function and is part of the ClojureScript runtime (standard library), and 1 is the first argument for the inc function.

(+ 1 2 3)
;; => 6

The + symbol represents an add function. It allows multiple parameters, whereas in ALGOL-type languages, + is an operator and only allows two parameters.

The prefix notation has huge advantages, some of them not always obvious. ClojureScript does not make a distinction between a function and an operator; everything is a function. The immediate advantage is that the prefix notation allows an arbitrary number of arguments per "operator". It also completely eliminates the problem of operator precedence.

3.4.2. Defining your own functions

You can define an unnamed (anonymous) function with the fn special form. This is one type of function definition; in the following example, the function takes two parameters and returns their average.

(fn [param1 param2]
  (/ (+ param1 param2) 2.0))

You can define a function and call it at the same time (in a single expression):

((fn [x] (* x x)) 5)
;; => 25

Let’s start creating named functions. But what does a named function really mean? It is very simple; in ClojureScript, functions are first-class and behave like any other value, so naming a function is done by simply binding the function to a symbol:

(def square (fn [x] (* x x)))

(square 12)
;; => 144

ClojureScript also offers the defn macro as a little syntactic sugar for making function definition more idiomatic:

(defn square
  "Return the square of a given number."
  [x]
  (* x x))

The string that comes between the function name and the parameter vector is called a docstring (documentation string); programs that automatically create web documentation from your source files will use these docstrings.

3.4.3. Functions with multiple arities

ClojureScript also comes with the ability to define functions with an arbitrary number of arguments. (The term arity means the number of arguments that a function takes.) The syntax is almost the same as for defining an ordinary function, with the difference that it has more than one body.

Let’s see an example, which will explain it better:

(defn myinc
  "Self defined version of parameterized `inc`."
  ([x] (myinc x 1))
  ([x increment]
   (+ x increment)))

This line: ([x] (myinc x 1)) says that if there is only one argument, call the function myinc with that argument and the number 1 as the second argument. The other function body ([x increment] (+ x increment)) says that if there are two arguments, return the result of adding them.

Here are some examples using the previously defined multi-arity function. Observe that if you call a function with the wrong number of arguments, the compiler will emit an error message.

(myinc 1)
;; => 2

(myinc 1 3)
;; => 4

(myinc 1 3 3)
;; Compiler error
Note
Explaining the concept of "arity" is out of the scope of this book, however you can read about that in this Wikipedia article.

3.4.4. Variadic functions

Another way to accept multiple parameters is defining variadic functions. Variadic functions are functions that accept an arbitrary number of arguments:

(defn my-variadic-set
  [& params]
  (set params))

(my-variadic-set 1 2 3 1)
;; => #{1 2 3}

The way to denote a variadic function is using the & symbol prefix on its arguments vector.

The variadic function can also be combined with multiple arities:

(defn max
  ([a] a)
  ([a b] (special-max a b))
  ([a b c] (special-max a b c))
  ([a b c d] (special-max a b c d))
  ([a b c d e] (special-max a b c d e))
  ([a b c d e f] (special-max a b c d e f))
  ([a b c d e f & other]
   (reduce max (special-max a b c d e f) other)))

You may wonder why I really need this syntax and define all that arities. Well, sometimes it is for documentation, since it prefers to name the first required parameters and leave the rest in a sequence. It is also a performance issue, since calling an arity with fixed variables will always be faster than calling a variadic function.

Note
special-max is an invented function, just for make an example.

In the function parameters you can apply destructuring, which is explained in detail later.

3.4.5. Named Parameters

Parameters by name in ClojureScript is nothing more than a special case of destructuring. See the destructuring section to better understand all the concepts.

There are times when we want to pass optional parameters to a function, something like options and it is better to give those options a name to be concise:

(defn my-prefix
  [value & {:keys [default] :or {default "foo-"}}]
  (str default value))

When you call this function, you just can omit the options, and the default will be used:

(my-prefix "hello")
;; => "foo-hello"

Then, provide a different prefix using named parameters:

(my-prefix "hello" :default "bar-")
;; => "bar-hello"

The good part is that the name parameters do not impose a fixed calling convention. Nothing really prevents you from calling that function with a map as a second parameter with all the options you want to pass.

(my-prefix "hello" {:default "bar-"})
;; => "bar-hello"

3.4.6. Short syntax for anonymous functions

ClojureScript provides a shorter syntax for defining anonymous functions using the #() reader macro (usually leads to one-liners). Reader macros are "special" expressions that will be transformed to the appropriate language form at compile time; in this case, to some expression that uses the fn special form.

(def average #(/ (+ %1 %2) 2))

(average 3 4)
;; => 3.5

The preceding definition is shorthand for:

(def average-longer (fn [a b] (/ (+ a b) 2)))

(average-longer 7 8)
;; => 7.5

The %1, %2…​ %N are simple markers for parameter positions that are implicitly declared when the reader macro will be interpreted and converted to a fn expression.

If a function only accepts one argument, you can omit the number after the % symbol, e.g., a function that squares a number: #(* %1 %1)) can be written #(* % %)).

Additionally, this syntax also supports the variadic form with the %& symbol:

(def my-variadic-set #(set %&))

(my-variadic-set 1 2 2)
;; => #{1 2}

This way of defining functions is generally useful for defining single line functions.

3.5. Flow control

ClojureScript has a very different approach to flow control than languages like JavaScript, C, etc. In an expression based language as it is CLJS, flow control structures also returns values.

3.5.1. Branching with if

Let’s start with a basic one: if. In ClojureScript, the if is an expression and not a statement, and it has three parameters: the first one is the condition expression, the second one is an expression that will be evaluated if the condition expression evaluates to logical true, and the third expression will be evaluated otherwise. The else part is optional.

(defn discount
  "You get 5% discount for ordering 100 or more items"
  [quantity]
  (if (>= quantity 100)
    0.05
    0))

(discount 30)
;; => 0

(discount 130)
;; => 0.05

The block expression do can be used to have multiple expressions in an if branch. do is explained in the next section.

3.5.2. Branching with cond

Sometimes, the if expression can be slightly limiting because it does not have the "else if" part to add more than one condition. The cond macro comes to the rescue.

With the cond expression, you can define multiple conditions:

(defn mypos?
  [x]
  (cond
    (> x 0) "positive"
    (< x 0) "negative"
    :else "zero"))

(mypos? 0)
;; => "zero"

(mypos? -2)
;; => "negative"

The :else keyword has no special meaning in cond. It just a truthy value and in reallity can be any keyword os anything that evaluates to a logical true.

Also, cond has another form, called condp, that works very similarly to the simple cond but looks cleaner when the condition (also called a predicate) is the same for all conditions:

(defn translate-lang-code
  [code]
  (condp = (keyword code)
    :es "Spanish"
    :en "English"
    "Unknown"))

(translate-lang-code "en")
;; => "English"

(translate-lang-code "fr")
;; => "Unknown"

The line condp = (keyword code) means that, in each of the following lines, ClojureScript will apply the = function to the result of evaluating (keyword code).

3.5.3. Branching with case

The case branching expression has a similar use as our previous example with condp. The main differences are that case always uses the = predicate/function and its branching values are evaluated at compile time. This results in a more performant form than cond or condp but has the disadvantage that the condition value must be static.

Here is the previous example rewritten to use case:

(defn translate-lang-code
  [code]
  (case code
    "es" "Spanish"
    "en" "English"
    "Unknown"))

(translate-lang-code "en")
;; => "English"

(translate-lang-code "fr")
;; => "Unknown"

3.6. Locals, Blocks, and Loops

3.6.1. Blocks

In JavaScript, braces { and } delimit a block of code that “belongs together”. Blocks in ClojureScript are created using the do expression and are usually used for side effects, like printing something to the console or writing a log in a logger.

A side effect is something that is not necessary for the return value.

The do expression accepts as its parameter an arbitrary number of other expressions, but it returns the return value only from the last one:

(do
  (println "hello world")
  (println "hola mundo")
  (* 3 5) ;; this value will not be returned; it is thrown away
  (+ 1 2))

;; hello world
;; hola mundo
;; => 3

3.6.2. Locals

ClojureScript does not have the concept of variables as in ALGOL-like languages, but it does have locals. Locals, as per usual, are immutable, and if you try to mutate them, the compiler will throw an error.

Locals are defined with the let expression. The expression starts with a vector as the first parameter followed by an arbitrary number of expressions. The first parameter (the vector) should contain an arbitrary number of pairs that give a binding form (usually a symbol) followed by an expression whose value will be bound to this new local for the remainder of the let expression.

(let [x (inc 1)
      y (+ x 1)]
  (println "Simple message from the body of a let")
  (* x y))
;; Simple message from the body of a let
;; => 6

In the preceding example, the symbol x is bound to the value of executing the (inc 1) expression, which comes out to 2, and the symbol y is bound to the sum of x and 1, which comes out to 3. Given those bindings, the expressions (println "Simple message from the body of a let") and (* x y) are evaluated.

The body of the let expression has an implicit do, so you can put there multiple expressions, where the last one will be used as return value.

3.6.3. Loops

The functional approach of ClojureScript means that it does not have standard, well-known, statement-based loops such as for in JavaScript. The loops in ClojureScript are handled using recursion. Recursion sometimes requires additional thinking about how to model your problem in a slightly different way than imperative languages.

Many of the common patterns for which for is used in other languages are achieved through higher-order functions - functions that accept other functions as parameters.

Looping with loop/recur

Let’s take a look at how to express loops using recursion with the loop and recur forms. loop defines a possibly empty list of bindings (notice the symmetry with let) and recur jumps execution back to the looping point with new values for those bindings.

Let’s see an example:

(loop [x 0]
  (println "Looping with " x)
  (if (= x 2)
    (println "Done looping!")
    (recur (inc x))))
;; Looping with 0
;; Looping with 1
;; Looping with 2
;; Done looping!
;; => nil

In the above snippet, we bind the name x to the value 0 and execute the body. Since the condition is not met the first time, it’s rerun with recur, incrementing the binding value with the inc function. We do this once more until the condition is met and, since there aren’t any more recur calls, exit the loop.

Note that loop isn’t the only point we can recur to; using recur inside a function executes the body of the function recursively with the new bindings:

(defn recursive-function
  [x]
  (println "Looping with" x)
  (if (= x 2)
    (println "Done looping!")
    (recur (inc x))))

(recursive-function 0)
;; Looping with 0
;; Looping with 1
;; Looping with 2
;; Done looping!
;; => nil
Replacing for loops with higher-order functions

In imperative programming languages it is common to use for loops to iterate over data and transform it, usually with the intent being one of the following:

  • Transform every value in the iterable yielding another iterable

  • Filter the elements of the iterable by certain criteria

  • Convert the iterable to a value where each iteration depends on the result from the previous one

  • Run a computation for every value in the iterable

The above actions are encoded in higher-order functions and syntactic constructs in ClojureScript; let’s see an example of the first three.

For transforming every value in an iterable data structure we use the map function, which takes a function and a sequence and applies the function to every element:

(map inc [0 1 2])
;; => (1 2 3)

The first parameter for map can be any function that takes one argument and returns a value. For example, if you had a graphing application and you wanted to graph the equation y = 3x + 5 for a set of x values, you could get the y values like this:

(defn y-value [x] (+ (* 3 x) 5))

(map y-value [1 2 3 4 5])
;; => (8 11 14 17 20)

If your function is short, you can use an anonymous function instead, either the normal or short syntax:

(map (fn [x] (+ (* 3 x) 5)) [1 2 3 4 5])
;; => (8 11 14 17 20)

(map #(+ (* 3 %) 5) [1 2 3 4 5])
;; => (8 11 14 17 20)

For filtering the values of a data structure we use the filter function, which takes a predicate and a sequence and gives a new sequence with only the elements that returned true for the given predicate:

(filter odd? [1 2 3 4])
;; => (1 3)

Again, you can use any function that returns true or false as the first argument to filter. Here is an example that keeps only words less than five characters long. (The count function returns the length of its argument.)

(filter (fn [word] (< (count word) 5)) ["ant" "baboon" "crab" "duck" "echidna" "fox"])
;; => ("ant" "crab" "duck" "fox")

Converting an iterable to a single value, accumulating the intermediate result at every step of the iteration can be achieved with reduce, which takes a function for accumulating values, an optional initial value and a collection:

(reduce + 0 [1 2 3 4])
;; => 10

Yet again, you can provide your own function as the first argument to reduce, but your function must have two parameters. The first one is the "accumulated value" and the second parameter is the collection item being processed. The function returns a value that becomes the accumulator for the next item in the list. For example, here is how you would find the sum of squares of a set of numbers (this is an important calculation in statistics). Using a separate function:

(defn sum-squares
  [accumulator item]
  (+ accumulator (* item item)))

(reduce sum-squares 0 [3 4 5])
;; => 50

…​and with an anonymous function:

(reduce (fn [acc item] (+ acc (* item item))) 0 [3 4 5])
;; => 50

Here is a reduce that finds the total number of characters in a set of words:

(reduce (fn [acc word] (+ acc (count word))) 0 ["ant" "bee" "crab" "duck"])
;; => 14

We have not used the short syntax here because, although it requires less typing, it can be less readable, and when you are starting with a new language, it’s important to be able to read what you wrote! If you are comfortable with the short syntax, feel free to use it.

Remember to choose your starting value for the accumulator carefully. If you wanted to use reduce to find the product of a series of numbers, you would have to start with one rather than zero, otherwise all the numbers would be multiplied by zero!

;; wrong starting value
(reduce * 0 [3 4 5])
;; => 0

;; correct starting accumulator
(reduce * 1 [3 4 5])
;; => 60
for sequence comprehensions

In ClojureScript, the for construct isn’t used for iteration but for generating sequences, an operation also known as "sequence comprehension". In this section we’ll learn how it works and use it to declaratively build sequences.

for takes a vector of bindings and an expression and generates a sequence of the result of evaluating the expression. Let’s take a look at an example:

(for [x [1 2 3]]
  [x (* x x)])
;; => ([1 1] [2 4] [3 9])

In this example, x is bound to each of the items in the vector [1 2 3] in turn, and returns a new sequence of two-item vectors with the original item squared.

for supports multiple bindings, which will cause the collections to be iterated in a nested fashion, much like nesting for loops in imperative languages. The innermost binding iterates “fastest.”

(for [x [1 2 3]
      y [4 5]]
  [x y])

;; => ([1 4] [1 5] [2 4] [2 5] [3 4] [3 5])

We can also follow the bindings with three modifiers: :let for creating local bindings, :while for breaking out of the sequence generation, and :when for filtering out values.

Here’s an example of local bindings using the :let modifier; note that the bindings defined with it will be available in the expression:

(for [x [1 2 3]
      y [4 5]
      :let [z (+ x y)]]
  z)
;; => (5 6 6 7 7 8)

We can use the :while modifier for expressing a condition that, when it is no longer met, will stop the sequence generation. Here’s an example:

(for [x [1 2 3]
      y [4 5]
      :while (= y 4)]
  [x y])

;; => ([1 4] [2 4] [3 4])

For filtering out generated values, use the :when modifier as in the following example:

(for [x [1 2 3]
      y [4 5]
      :when (= (+ x y) 6)]
  [x y])

;; => ([1 5] [2 4])

We can combine the modifiers shown above for expressing complex sequence generations or more clearly expressing the intent of our comprehension:

(for [x [1 2 3]
      y [4 5]
      :let [z (+ x y)]
      :when (= z 6)]
  [x y])

;; => ([1 5] [2 4])

When we outlined the most common usages of the for construct in imperative programming languages, we mentioned that sometimes we want to run a computation for every value in a sequence, not caring about the result. Presumably we do this for achieving some sort of side-effect with the values of the sequence.

ClojureScript provides the doseq construct, which is analogous to for but executes the expression, discards the resulting values, and returns nil.

(doseq [x [1 2 3]
        y [4 5]
       :let [z (+ x y)]]
  (println x "+" y "=" z))

;; 1 + 4 = 5
;; 1 + 5 = 6
;; 2 + 4 = 6
;; 2 + 5 = 7
;; 3 + 4 = 7
;; 3 + 5 = 8
;; => nil

If you want just iterate and apply some side effectfull operation (like println) over each item in the collection, you can just use the specialized function run! that internally uses fast reduction:

(run! println [1 2 3])
;; 1
;; 2
;; 3
;; => nil

This function explicitly returns nil.

3.7. Collection types

3.7.1. Immutable and persistent

We mentioned before that ClojureScript collections are persistent and immutable, but we didn’t explain what that meant.

An immutable data structure, as its name suggests, is a data structure that cannot be changed. In-place updates are not allowed in immutable data structures.

Let’s illustrate that with an example: appending values to a vector using the conj (conjoin) operation.

(let [xs [1 2 3]
      ys (conj xs 4)]
  (println "xs:" xs)
  (println "ys:" ys))

;; xs: [1 2 3]
;; ys: [1 2 3 4]
;; => nil

As you can see, we derived a new version of the xs vector appending an element to it and got a new vector ys with the element added. However, the xs vector remained unchanged because it is immutable.

A persistent data structure is a data structure that returns a new version of itself when transforming it, leaving the original unmodified. ClojureScript makes this memory and time efficient using an implementation technique called structural sharing, where most of the data shared between two versions of a value is not duplicated and transformations of a value are implemented by copying the minimal amount of data required.

If you want to see an example of how structural sharing works, read on. If you’re not interested in more details you can skip over to the next section.

For illustrating the structural sharing of ClojureScript data structures, let’s compare whether some parts of the old and new versions of a data structure are actually the same object with the identical? predicate. We’ll use the list data type for this purpose:

(let [xs (list 1 2 3)
      ys (cons 0 xs)]
  (println "xs:" xs)
  (println "ys:" ys)
  (println "(rest ys):" (rest ys))
  (identical? xs (rest ys)))

;; xs: (1 2 3)
;; ys: (0 1 2 3)
;; (rest ys): (1 2 3)
;; => true

As you can see in the example, we used cons (construct) to prepend a value to the xs list and we got a new list ys with the element added. The rest of the ys list (all the values but the first) are the same object in memory as the xs list, thus xs and ys share structure.

3.7.2. The sequence abstraction

One of the central ClojureScript abstractions is the sequence which can be thought of as a list and can be derived from any of the collection types. It is persistent and immutable like all collection types, and many of the core ClojureScript functions return sequences.

The types that can be used to generate a sequence are called "seqables"; we can call seq on them and get a sequence back. Sequences support two basic operations: first and rest. They both call seq on the argument we provide them:

(first [1 2 3])
;; => 1

(rest [1 2 3])
;; => (2 3)

Calling seq on a seqable can yield different results if the seqable is empty or not. It will return nil when empty and a sequence otherwise:

(seq [])
;; => nil

(seq [1 2 3])
;; => (1 2 3)

next is a similar sequence operation to rest, but it differs from the latter in that it yields a nil value when called with a sequence with one or zero elements. Note that, when given one of the aforementioned sequences, the empty sequence returned by rest will evaluate as a boolean true whereas the nil value returned by next will evaluate as false (see the section on booleans and truthiness).

(rest [])
;; => ()

(next [])
;; => nil

(rest [1 2 3])
;; => (2 3)

(next [1 2 3])
;; => (2 3)
nil-punning

Since seq returns nil when the collection is empty, and nil evaluates to false in boolean context, you can check to see if a collection is empty by using the seq function. The technical term for this is nil-punning.

(defn print-coll
  [coll]
  (when (seq coll)
    (println "Saw " (first coll))
    (recur (rest coll))))

(print-coll [1 2 3])
;; Saw 1
;; Saw 2
;; Saw 3
;; => nil

(print-coll #{1 2 3})
;; Saw 1
;; Saw 3
;; Saw 2
;; => nil

Though nil is neither a seqable nor a sequence, it is supported by all the functions we saw so far:

(seq nil)
;; => nil

(first nil)
;; => nil

(rest nil)
;; => ()
Functions that work on sequences

The ClojureScript core functions for transforming collections make sequences out of their arguments and are implemented in terms of the generic sequence operations we learned about in the preceding section. This makes them highly generic because we can use them on any data type that is seqable. Let’s see how we can use map with a variety of seqables:

(map inc [1 2 3])
;; => (2 3 4)

(map inc #{1 2 3})
;; => (2 4 3)

(map count {:a 41 :b 40})
;; => (2 2)

(map inc '(1 2 3))
;; => (2 3 4)
Note
When you use the map function on a map collection, your higher-order function will receive a two-item vector containing a key and value from the map. The following example uses destructuring to access the key and value.
(map (fn [[key value]] (* value value))
     {:ten 10 :seven 7 :four 4})
;; => (100 49 16)

Obviously the same operation can be done in more idiomatic way only obtaining a seq of values:

(map (fn [value] (* value value))
     (vals {:ten 10 :seven 7 :four 4}))
;; => (100 49 16)

As you may have noticed, functions that operate on sequences are safe to use with empty collections or even nil values since they don’t need to do anything but return an empty sequence when encountering such values.

(map inc [])
;; => ()

(map inc #{})
;; => ()

(map inc nil)
;; => ()

We already saw examples with the usual suspects like map, filter, and reduce, but ClojureScript offers a plethora of generic sequence operations in its core namespace. Note that many of the operations we’ll learn about either work with seqables or are extensible to user-defined types.

We can query a value to know whether it’s a collection type with the coll? predicate:

(coll? nil)
;; => false

(coll? [1 2 3])
;; => true

(coll? {:language "ClojureScript" :file-extension "cljs"})
;; => true

(coll? "ClojureScript")
;; => false

Similar predicates exist for checking if a value is a sequence (with seq?) or a seqable (with seqable?):

(seq? nil)
;; => false
(seqable? nil)
;; => false

(seq? [])
;; => false
(seqable? [])
;; => true

(seq? #{1 2 3})
;; => false
(seqable? #{1 2 3})
;; => true

(seq? "ClojureScript")
;; => false
(seqable? "ClojureScript")
;; => false

For collections that can be counted in constant time, we can use the count operation. This operation also works on strings, even though, as you have seen, they are not collections, sequences, or seqable.

(count nil)
;; => 0

(count [1 2 3])
;; => 3

(count {:language "ClojureScript" :file-extension "cljs"})
;; => 2

(count "ClojureScript")
;; => 13

It must be taken into account that there are data types such as lists or lazy lists that do not have a defined size, in this case when calling count on them, it will linearly traverse the list to know how many elements it has and this can have an impact on the performance.

We can also get an empty variant of a given collection with the empty function:

(empty nil)
;; => nil

(empty [1 2 3])
;; => []

(empty #{1 2 3})
;; => #{}

The empty? predicate returns true if the given collection is empty:

(empty? nil)
;; => true

(empty? [])
;; => true

(empty? #{1 2 3})
;; => false

The conj operation adds elements to collections and may add them in different "places" depending on the type of collection. It adds them where it is most performant for the collection type, but note that not every collection has a defined order.

We can pass as many elements as we want to add to conj; let’s see it in action:

(conj nil 42)
;; => (42)

(conj [1 2] 3)
;; => [1 2 3]

(conj [1 2] 3 4 5)
;; => [1 2 3 4 5]

(conj '(1 2) 0)
;; => (0 1 2)

(conj #{1 2 3} 4)
;; => #{1 3 2 4}

(conj {:language "ClojureScript"} [:file-extension "cljs"])
;; => {:language "ClojureScript", :file-extension "cljs"}
Laziness

Most of ClojureScript’s sequence-returning functions generate lazy sequences instead of eagerly creating a whole new sequence. Lazy sequences generate their contents as they are requested, usually when iterating over them. Laziness ensures that we don’t do more work than we need to and gives us the possibility of treating potentially infinite sequences as regular ones.

Consider the range function, which generates a range of integers:

(range 5)
;; => (0 1 2 3 4)
(range 1 10)
;; => (1 2 3 4 5 6 7 8 9)
(range 10 100 15)
;; (10 25 40 55 70 85)

If you just say (range), you will get an infinite sequence of all the integers. Do not try this in the REPL, unless you are prepared to wait for a very, very long time, because the REPL wants to fully evaluate the expression.

Here is a contrived example. Let’s say you are writing a graphing program and you are graphing the equation y= 2 x 2 + 5, and you want only those values of x for which the y value is less than 100. You can generate all the numbers 0 through 100, which will certainly be enough, and then take-while the condition holds:

(take-while (fn [x] (< (+ (* 2 x x) 5) 100))
            (range 0 100))
;; => (0 1 2 3 4 5 6)

When we have used map and filter in the previous examples, the result when printed on the screen is evaluated to see the return value, but if we save the content without printing it, no operation will be executed until we request the first element of the sequence.

3.7.3. Collections in depth

Now that we’re acquainted with ClojureScript’s sequence abstraction and some of the generic sequence manipulating functions, it’s time to dive into the concrete collection types and the operations they support.

Lists

In ClojureScript, lists are mostly used as a data structure for grouping symbols together into programs. Unlike in other Lisps, many of the syntactic constructs of ClojureScript use data structures different from the list (vectors and maps). This makes code less uniform, but the gains in readability are well worth the price.

You can think of ClojureScript lists as singly linked lists, where each node contains a value and a pointer to the rest of the list. This makes it natural (and fast!) to add items to the front of the list, since adding to the end would require traversal of the entire list. The prepend operation is performed using the cons function.

(cons 0 (cons 1 (cons 2 ())))
;; => (0 1 2)

We used the literal () to represent the empty list. Since it doesn’t contain any symbols, it is not treated as a function call. However, when using list literals that contain elements, we need to quote them to prevent ClojureScript from evaluating them as a function call:

(cons 0 '(1 2))
;; => (0 1 2)

Since the head is the position that has constant time addition in the list collection, the conj operation on lists naturally adds items to the front:

(conj '(1 2) 0)
;; => (0 1 2)

Lists and other ClojureScript data structures can be used as stacks using the peek, pop, and conj functions. Note that the top of the stack will be the "place" where conj adds elements, making conj equivalent to the stack’s push operation. In the case of lists, conj adds elements to the front of the list, peek returns the first element of the list, and pop returns a list with all the elements but the first one.

Note that the two operations that return a stack (conj and pop) don’t change the type of the collection used for the stack.

(def list-stack '(0 1 2))

(peek list-stack)
;; => 0

(pop list-stack)
;; => (1 2)

(type (pop list-stack))
;; => cljs.core/List

(conj list-stack -1)
;; => (-1 0 1 2)

(type (conj list-stack -1))
;; => cljs.core/List

One thing that lists are not particularly good at is random indexed access. Since they are stored in a single linked list-like structure in memory, random access to a given index requires a linear traversal in order to either retrieve the requested item or throw an index out of bounds error. Non-indexed ordered collections like lazy sequences also suffer from this limitation.

Vectors

Vectors are one of the most common data structures in ClojureScript. They are used as a syntactic construct in many places where more traditional Lisps use lists, for example in function argument declarations and let bindings.

ClojureScript vectors have enclosing brackets [] in their syntax literals. They can be created with vector and from another collection with vec:

(vector? [0 1 2])
;; => true

(vector 0 1 2)
;; => [0 1 2]

(vec '(0 1 2))
;; => [0 1 2]

Vectors are, like lists, ordered collections of heterogeneous values. Unlike lists, vectors grow naturally from the tail, so the conj operation appends items to the end of a vector. Insertion on the end of a vector is effectively constant time:

(conj [0 1] 2)
;; => [0 1 2]

Another thing that differentiates lists and vectors is that vectors are indexed collections and as such support efficient random index access and non-destructive updates. We can use the nth function to retrieve values given an index:

(nth [0 1 2] 0)
;; => 0

Since vectors associate sequential numeric keys (indexes) to values, we can treat them as an associative data structure. ClojureScript provides the assoc function that, given an associative data structure and a set of key-value pairs, yields a new data structure with the values corresponding to the keys modified. Indexes begin at zero for the first element in a vector.

(assoc ["cero" "uno" "two"] 2 "dos")
;; => ["cero" "uno" "dos"]

Note that we can only assoc to a key that is either contained in the vector already or if it is the last position in a vector:

(assoc ["cero" "uno" "dos"] 3 "tres")
;; => ["cero" "uno" "dos" "tres"]

(assoc ["cero" "uno" "dos"] 4 "cuatro")
;; Error: Index 4 out of bounds [0,3]

Perhaps surprisingly, associative data structures can also be used as functions. They are functions of their keys to the values they are associated with. In the case of vectors, if the given key is not present an exception is thrown:

(["cero" "uno" "dos"] 0)
;; => "cero"

(["cero" "uno" "dos"] 2)
;; => "dos"

(["cero" "uno" "dos"] 3)
;; Error: Not item 3 in vector of length 3

As with lists, vectors can also be used as stacks with the peek, pop, and conj functions. Note, however, that vectors grow from the opposite end of the collection as lists:

(def vector-stack [0 1 2])

(peek vector-stack)
;; => 2

(pop vector-stack)
;; => [0 1]

(type (pop vector-stack))
;; => cljs.core/PersistentVector

(conj vector-stack 3)
;; => [0 1 2 3]

(type (conj vector-stack 3))
;; => cljs.core/PersistentVector

The map and filter operations return lazy sequences, but as it is common to need a fully realized sequence after performing those operations, vector-returning counterparts of such functions are available as mapv and filterv. They have the advantages of being faster than building a vector from a lazy sequence and making your intent more explicit:

(map inc [0 1 2])
;; => (1 2 3)

(type (map inc [0 1 2]))
;; => cljs.core/LazySeq

(mapv inc [0 1 2])
;; => [1 2 3]

(type (mapv inc [0 1 2]))
;; => cljs.core/PersistentVector
Maps

Maps are ubiquitous in ClojureScript. Like vectors, they are also used as a syntactic construct, particularly for attaching metadata to vars. Any ClojureScript data structure can be used as a key in a map, although it’s common to use keywords since they can also be called as functions.

ClojureScript maps are written literally as key-value pairs enclosed in braces {}. Alternatively, they can be created with the hash-map function:

(map? {:name "Cirilla"})
;; => true

(hash-map :name "Cirilla")
;; => {:name "Cirilla"}

(hash-map :name "Cirilla" :surname "Fiona")
;; => {:name "Cirilla" :surname "Fiona"}

Since regular maps don’t have a specific order, the conj operation just adds one or more key-value pairs to a map. conj for maps expects one or more sequences of key-value pairs as its last arguments:

(def ciri {:name "Cirilla"})

(conj ciri [:surname "Fiona"])
;; => {:name "Cirilla", :surname "Fiona"}

(conj ciri [:surname "Fiona"] [:occupation "Wizard"])
;; => {:name "Cirilla", :surname "Fiona", :occupation "Wizard"}

In the preceding example, it just so happens that the order was preserved, but if you have many keys, you will see that the order is not preserved.

Maps associate keys to values and, as such, are an associative data structure. They support adding associations with assoc and, unlike vectors, removing them with dissoc. assoc will also update the value of an existing key. Let’s explore these functions:

(assoc {:name "Cirilla"} :surname "Fiona")
;; => {:name "Cirilla", :surname "Fiona"}
(assoc {:name "Cirilla"} :name "Alfonso")
;; => {:name "Alfonso"}
(dissoc {:name "Cirilla"} :name)
;; => {}

Maps are also functions of their keys, returning the values related to the given keys. Unlike vectors, they return nil if we supply a key that is not present in the map:

({:name "Cirilla"} :name)
;; => "Cirilla"

({:name "Cirilla"} :surname)
;; => nil

ClojureScript also offers sorted hash maps which behave like their unsorted versions but preserve order when iterating over them. We can create a sorted map with default ordering with sorted-map:

(def sm (sorted-map :c 2 :b 1 :a 0))
;; => {:a 0, :b 1, :c 2}

(keys sm)
;; => (:a :b :c)

If we need a custom ordering we can provide a comparator function to sorted-map-by, let’s see an example inverting the value returned by the built-in compare function. Comparator functions take two items to compare and return -1 (if the first item is less than the second), 0 (if they are equal), or 1 (if the first item is greater than the second).

(defn reverse-compare [a b] (compare b a))

(def sm (sorted-map-by reverse-compare :a 0 :b 1 :c 2))
;; => {:c 2, :b 1, :a 0}

(keys sm)
;; => (:c :b :a)
Sets

Sets in ClojureScript have literal syntax as values enclosed in #{} and they can be created with the set constructor. They are unordered collections of values without duplicates.

(set? #{\a \e \i \o \u})
;; => true

(set [1 1 2 3])
;; => #{1 2 3}

Set literals cannot contain duplicate values. If you accidentally write a set literal with duplicates an error will be thrown:

#{1 1 2 3}
;; clojure.lang.ExceptionInfo: Duplicate key: 1

There are many operations that can be performed with sets, although they are located in the clojure.set namespace and thus need to be imported. You’ll learn the details of namespacing later; for now, you only need to know that we are loading a namespace called clojure.set and binding it to the s symbol.

(require '[clojure.set :as s])

(def danish-vowels #{\a \e \i \o \u   })
;; => #{"a" "e" "å" "æ" "i" "o" "u" "ø"}

(def spanish-vowels #{\a \e \i \o \u})
;; => #{"a" "e" "i" "o" "u"}

(s/difference danish-vowels spanish-vowels)
;; => #{"å" "æ" "ø"}

(s/union danish-vowels spanish-vowels)
;; => #{"a" "e" "å" "æ" "i" "o" "u" "ø"}

(s/intersection danish-vowels spanish-vowels)
;; => #{"a" "e" "i" "o" "u"}

A nice property of immutable sets is that they can be nested. Languages that have mutable sets can end up containing duplicate values, but that can’t happen in ClojureScript. In fact, all ClojureScript data structures can be nested arbitrarily due to immutability.

Sets also support the generic conj operation just like every other collection does.

(def spanish-vowels #{\a \e \i \o \u})
;; => #{"a" "e" "i" "o" "u"}

(def danish-vowels (conj spanish-vowels   ))
;; => #{"a" "e" "i" "o" "u" "æ" "ø" "å"}

(conj #{1 2 3} 1)
;; => #{1 3 2}

Sets act as read-only associative data that associates the values it contains to themselves. Since every value except nil and false is truthy in ClojureScript, we can use sets as predicate functions:

(def vowels #{\a \e \i \o \u})
;; => #{"a" "e" "i" "o" "u"}

(get vowels \b)
;; => nil

(contains? vowels \b)
;; => false

(vowels \a)
;; => "a"

(vowels \z)
;; => nil

(filter vowels "Hound dog")
;; => ("o" "u" "o")

Sets have a sorted counterpart like maps do that are created using the functions sorted-set and sorted-set-by which are analogous to map’s sorted-map and sorted-map-by.

(def unordered-set #{[0] [1] [2]})
;; => #{[0] [2] [1]}

(seq unordered-set)
;; => ([0] [2] [1])

(def ordered-set (sorted-set [0] [1] [2]))
;; =># {[0] [1] [2]}

(seq ordered-set)
;; => ([0] [1] [2])
Queues

ClojureScript also provides a persistent and immutable queue. Queues are not used as pervasively as other collection types. They can be created using the #queue [] literal syntax, but there are no convenient constructor functions for them.

(def pq #queue [1 2 3])
;; => #queue [1 2 3]

Using conj to add values to a queue adds items onto the rear:

(def pq #queue [1 2 3])
;; => #queue [1 2 3]

(conj pq 4 5)
;; => #queue [1 2 3 4 5]

A thing to bear in mind about queues is that the stack operations don’t follow the usual stack semantics (pushing and popping from the same end). pop takes values from the front position, and conj pushes (appends) elements to the back.

(def pq #queue [1 2 3])
;; => #queue [1 2 3]

(peek pq)
;; => 1

(pop pq)
;; => #queue [2 3]

(conj pq 4)
;; => #queue [1 2 3 4]

Queues are not as frequently used as lists or vectors, but it is good to know that they are available in ClojureScript, as they may occasionally come in handy.

3.8. Destructuring

Destructuring, as its name suggests, is a way of taking apart structured data such as collections and focusing on individual parts of them. ClojureScript offers a concise syntax for destructuring both indexed sequences and associative data structures that can be used any place where bindings are declared.

Let’s see an example of what destructuring is useful for that will help us understand the previous statements better. Imagine that you have a sequence but are only interested in the first and third item. You could get a reference to them easily with the nth function:

(let [v [0 1 2]
      fst (nth v 0)
      thrd (nth v 2)]
  [thrd fst])
;; => [2 0]

However, the previous code is overly verbose. Destructuring lets us extract values of indexed sequences more succintly using a vector on the left-hand side of a binding:

(let [[fst _ thrd] [0 1 2]]
  [thrd fst])
;; => [2 0]

In the above example, [fst _ thrd] is a destructuring form. It is represented as a vector and used for binding indexed values to the symbols fst and thrd, corresponding to the index 0 and 2, respectively. The _ symbol is used as a placeholder for indexes we are not interested in — in this case 1.

Note that destructuring is not limited to the let binding form; it works in almost every place where we bind values to symbols such as in the for and doseq special forms or in function arguments. We can write a function that takes a pair and swaps its positions very concisely using destructuring syntax in function arguments:

(defn swap-pair [[fst snd]]
  [snd fst])

(swap-pair [1 2])
;; => [2 1]

(swap-pair '(3 4))
;; => [4 3]

Positional destructuring with vectors is quite handy for taking indexed values out of sequences, but sometimes we don’t want to discard the rest of the elements in the sequence when destructuring. Similarly to how & is used for accepting variadic function arguments, the ampersand can be used inside a vector destructuring form for grouping together the rest of a sequence:

(let [[fst snd & more] (range 10)]
  {:first fst
   :snd snd
   :rest more})
;; => {:first 0, :snd 1, :rest (2 3 4 5 6 7 8 9)}

Notice how the value in the 0 index got bound to fst, the value in the 1 index got bound to snd, and the sequence of elements from 2 onwards got bound to the more symbol.

We may still be interested in a data structure as a whole even when we are destructuring it. This can be achieved with the :as keyword. If used inside a destructuring form, the original data structure is bound to the symbol following that keyword:

(let [[fst snd & more :as original] (range 10)]
  {:first fst
   :snd snd
   :rest more
   :original original})
;; => {:first 0, :snd 1, :rest (2 3 4 5 6 7 8 9), :original (0 1 2 3 4 5 6 7 8 9)}

Not only can indexed sequences be destructured, but associative data can also be destructured. Its destructuring binding form is represented as a map instead of a vector, where the keys are the symbols we want to bind values to and the values are the keys that we want to look up in the associative data structure. Let’s see an example:

(let [{language :language} {:language "ClojureScript"}]
  language)
;; => "ClojureScript"

In the above example, we are extracting the value associated with the :language key and binding it to the language symbol. When looking up keys that are not present, the symbol will get bound to nil:

(let [{name :name} {:language "ClojureScript"}]
  name)
;; => nil

Associative destructuring lets us give default values to bindings which will be used if the key isn’t found in the data structure we are taking apart. A map following the :or keyword is used for default values as the following examples show:

(let [{name :name :or {name "Anonymous"}} {:language "ClojureScript"}]
  name)
;; => "Anonymous"

(let [{name :name :or {name "Anonymous"}} {:name "Cirilla"}]
  name)
;; => "Cirilla"

Associative destructuring also supports binding the original data structure to a symbol placed after the :as keyword:

(let [{name :name :as person} {:name "Cirilla" :age 49}]
  [name person])
;; => ["Cirilla" {:name "Cirilla" :age 49}]

Keywords aren’t the only things that can be the keys of associative data structures. Numbers, strings, symbols and many other data structures can be used as keys, so we can destructure using those, too. Note that we need to quote the symbols to prevent them from being resolved as a var lookup:

(let [{one 1} {0 "zero" 1 "one"}]
  one)
;; => "one"

(let [{name "name"} {"name" "Cirilla"}]
  name)
;; => "Cirilla"

(let [{lang 'language} {'language "ClojureScript"}]
  lang)
;; => "ClojureScript"

Since the values corresponding to keys are usually bound to their equivalent symbol representation (for example, when binding the value of :language to the symbol language) and keys are usually keywords, strings, or symbols, ClojureScript offers shorthand syntax for these cases.

We’ll show examples of all of these, starting with destructuring keywords using :keys:

(let [{:keys [name surname]} {:name "Cirilla" :surname "Fiona"}]
  [name surname])
;; => ["Cirilla" "Fiona"]

As you can see in the example, if we use the :keys keyword and associate it with a vector of symbols in a binding form, the values corresponding to the keywordized version of the symbols will be bound to them. The {:keys [name surname]} destructuring is equivalent to {name :name surname :surname}, only shorter.

The string and symbol shorthand syntax works exactly like :keys, but using the :strs and :syms keywords respectively:

(let [{:strs [name surname]} {"name" "Cirilla" "surname" "Fiona"}]
  [name surname])
;; => ["Cirilla" "Fiona"]

(let [{:syms [name surname]} {'name "Cirilla" 'surname "Fiona"}]
  [name surname])
;; => ["Cirilla" "Fiona"]

If the map you want to destructure has namespaced keywords as keys, you also can do it using the keyword syntax inside :keys vector:

(let [{:keys [::name ::surname]} {::name "Cirilla" ::surname "Fiona"}]
  [name surname])
;; => ["Cirilla" "Fiona"]

An interesting property of destructuring is that we can nest destructuring forms arbitrarily, which makes code that accesses nested data on a collection very easy to understand, as it mimics the collection’s structure:

(let [{[fst snd] :languages} {:languages ["ClojureScript" "Clojure"]}]
  [snd fst])
;; => ["Clojure" "ClojureScript"]

3.9. Threading Macros

Threading macros, also known as arrow functions, enables one to write more readable code when multiple nested function calls are performed.

Imagine you have (f (g (h x))) where a function f receives as its first parameter the result of executing function g, repeated multiple times. With the most basic threading macro you can convert that into (-> x (h) (g) (f)) which is easier to read.

The result is syntactic sugar, because the arrow functions are defined as macros and it does not imply any runtime performance. The (-> x (h) (g) (f)) is automatically converted to (f (g (h x))) at compile time.

Take note that the parenthesis on h, g and f are optional, and can be ommited: (f (g (h x))) is the same as (-> x h g f).

3.9.1. -> (thread-first macro)

This is called thread first because it threads the first argument throught the different expressions as first arguments.

Using a more concrete example, this is how the code looks without using threading macros:

(def book {:name "Lady of the Lake"
           :readers 0})

(update (assoc book :age 1999) :readers inc)
;; => {:name "Lady of the lake" :age 1999 :readers 1}

We can rewrite that code to use the -> threading macro:

(-> book
    (assoc :age 1999)
    (update :readers inc))
;; => {:name "Lady of the lake" :age 1999 :readers 1}

This threading macro is especially useful for transforming data structures, because ClojureScript (and Clojure) functions for data structures transformations consistently uses the first argument to receive the data structure.

3.9.2. ->> (thread-last macro)

The main difference between the thread-last and thread-first macros is that instead of threading the first argument given as the first argument on the following expresions, it threads it as the last argument.

Let’s look at an example:

(def numbers [1 2 3 4 5 6 7 8 9 0])

(take 2 (filter odd? (map inc numbers)))
;; => (3 5)

The same code written using ->> threading macro:

(->> numbers
     (map inc)
     (filter odd?)
     (take 2))
;; => (3 5)

This threading macro is especially useful for transforming sequences or collections of data because ClojureScript functions that work with sequences and collections consistently use the last argument position to receive them.

3.9.3. as-> (thread-as macro)

Finally, there are cases where neither -> nor ->> are applicable. In these cases, you’ll need to use as->, the more flexible alternative, that allows you to thread into any argument position, not just the first or last.

It expects two fixed arguments and an arbitrary number of expressions. As with ->, the first argument is a value to be threaded through the following forms. The second argument is the name of a binding. In each of the subsequent forms, the bound name can be used for the prior expression’s result.

Let’s see an example:

(as-> numbers $
  (map inc $)
  (filter odd? $)
  (first $)
  (hash-map :result $ :id 1))
;; => {:result 3 :id 1}

3.9.4. some->, some->> (thread-some macros)

Two of the more specialized threading macros that ClojureScript comes with. They work in the same way as their analagous -> and ->> macros with the additional support for short-circuiting the expression if one of the expresions evaluates to nil.

Let’s see another example:

(some-> (rand-nth [1 nil])
        (inc))
;; => 2

(some-> (rand-nth [1 nil])
        (inc))
;; => nil

This is an easy way avoid null pointer exceptions.

3.9.5. cond->, cond->> (thread-cond macros)

The cond-> and cond->> macros are analogous to -> and ->> that offers the ability to conditionally skip some steps from the pipeline. Let see an example:

(defn describe-number
  [n]
  (cond-> []
    (odd? n) (conj "odd")
    (even? n) (conj "even")
    (zero? n) (conj "zero")
    (pos? n) (conj "positive")))

(describe-number 3)
;; => ["odd" "positive"]

(describe-number 4)
;; => ["even" "positive"]

The value threading only happens when the corresponding condition evaluates to logical true.

3.10. Namespaces

3.10.1. Defining a namespace

The namespace is ClojureScript’s fundamental unit of code modularity. Namespaces are analogous to Java packages or Ruby and Python modules and can be defined with the ns macro. If you have ever looked at a little bit of ClojureScript source, you may have noticed something like this at the beginning of the file:

(ns myapp.core
  "Some docstring for the namespace.")

(def x "hello")

Namespaces are dynamic, meaning you can create one at any time. However, the convention is to have one namespace per file. Naturally, a namespace definition is usually at the beginning of the file, followed by an optional docstring.

Previously we have explained vars and symbols. Every var that you define will be associated with its namespace. If you do not define a concrete namespace, then the default one called "cljs.user" will be used:

(def x "hello")
;; => #'cljs.user/x

3.10.2. Loading other namespaces

Defining a namespace and the vars in it is really easy, but it’s not very useful if we can’t use symbols from other namespaces. For this purpose, the ns macro offers a simple way to load other namespaces.

Observe the following:

(ns myapp.main
  (:require myapp.core
            clojure.string))

(clojure.string/upper-case myapp.core/x)
;; => "HELLO"

As you can observe, we are using fully qualified names (namespace + var name) to access vars and functions from different namespaces.

While this will let you access other namespaces, it’s also repetitive and overly verbose. It will be especially uncomfortable if the name of a namespace is very long. To solve that, you can use the :as directive to create an additional (usually shorter) alias to the namespace. This is how it can be done:

(ns myapp.main
  (:require [myapp.core :as core]
            [clojure.string :as str]))

(str/upper-case core/x)
;; => "HELLO"

One peculiarity of the namespace aliases, is that they can be used to obtain namespaced keywords from a specific namespace:

(ns myapp.main
  (:require [myapp.core :as c]))

::c/foo
;; => :myapp.core/foo

In the same way, you can namespace all the keys on the moment of creation of a map:

(def x #::c {:a 1})

x
;; => #:myapp.core{:a 1}

(::c/a x)
;; => 1

Additionally, ClojureScript offers a simple way to refer to specific vars or functions from a concrete namespace using the :refer directive, followed by a sequence of symbols that will refer to vars in the namespace. Effectively, it’s as if those vars and functions are now part of your namespace, and you do not need to qualify them at all.

(ns myapp.main
  (:require [clojure.string :refer [upper-case]]))
(upper-case x)
;; => "HELLO"

And finally, you should know that everything located in the cljs.core namespace is automatically loaded and you should not require it explicitly. Sometimes you may want to declare vars that will clash with some others defined in the cljs.core namespace. To do this, the ns macro offers another directive that allows you to exclude specific symbols and prevent them from being automatically loaded.

Observe the following:

(ns myapp.main
  (:refer-clojure :exclude [min]))

(defn min
  [x y]
  (if (> x y)
    y
    x))

The ns macro also has other directives for loading host classes (with :import) and macros (with :refer-macros), but these are explained in other sections.

3.10.3. Namespaces and File Names

When you have a namespace like myapp.core, the code must be in a file named core.cljs inside the myapp directory. So, the preceding examples with namespaces myapp.core and myapp.main would be found in project with a file structure like this:

myapp
└── src
    └── myapp
        ├── core.cljs
        └── main.cljs

3.11. Abstractions and Polymorphism

I’m sure that at more than one time you have found yourself in this situation: you have defined a great abstraction (using interfaces or something similar) for your "business logic", and you have found the need to deal with another module over which you have absolutely no control, and you probably were thinking of creating adapters, proxies, and other approaches that imply a great amount of additional complexity.

Some dynamic languages allow "monkey-patching"; languages where the classes are open and any method can be defined and redefined at any time. Also, it is well known that this technique is a very bad practice.

You can’t trust languages that allow you to silently overwrite methods that you’re using when you import third party libraries; you cannot expect consistent behavior when this happens.

These symptoms are commonly called the "expression problem"; see http://en.wikipedia.org/wiki/Expression_problem for more details

3.11.1. Protocols

The ClojureScript primitive for defining "interfaces" is called a protocol. A protocol consists of a name and set of functions. All the functions have at least one argument corresponding to the this in JavaScript or self in Python.

Protocols provide a type-based polymorphism, and the dispatch is always done by the first argument (equivalent to JavaScript’s this, as previously mentioned).

A protocol looks like this:

(ns myapp.testproto)

(defprotocol IProtocolName
  "A docstring describing the protocol."
  (sample-method [this] "A doc string associated with this function."))
Note
the "I" prefix is commonly used to designate the separation of protocols and types. In the Clojure community, there are many different opinions about how the "I" prefix should be used. In our opinion, it is an acceptable solution to avoid name clashing and possible confusion. But omitting the prefix is not considered bad practice.

From the user perspective, protocol functions are simply plain functions defined in the namespace where the protocol is defined. This enables an easy and simple approach for avoiding conflicts between different protocols implemented for the same type that have conflicting function names.

Here’s an example. Let’s create a protocol called IInvertible for data that can be "inverted". It will have a single method named invert.

(defprotocol IInvertible
  "This is a protocol for data types that are 'invertible'"
  (invert [this] "Invert the given item."))
Extending existing types

One of the big strengths of protocols is the ability to extend existing, and possibly, third party types. This operation can be done in different ways.

Most of the time you will tend to use the extend-protocol or the extend-type macros to do this. This is how extend-type syntax looks:

(extend-type TypeA
  ProtocolA
  (function-from-protocol-a [this]
    ;; implementation here
    )

  ProtocolB
  (function-from-protocol-b-1 [this parameter1]
    ;; implementation here
    )
  (function-from-protocol-b-2 [this parameter1 parameter2]
    ;; implementation here
    ))

You can observe that with extend-type you are extending a single type with different protocols in a single expression.

Let’s play with our IInvertible protocol defined previously:

(extend-type string
  IInvertible
  (invert [this] (apply str (reverse this))))

(extend-type cljs.core.List
  IInvertible
  (invert [this] (reverse this)))

(extend-type cljs.core.PersistentVector
  IInvertible
  (invert [this] (into [] (reverse this))))

You may note that a special symbol string is used instead of js/String to extend the protocol for string. This is because the built-in javascript types have special treatment and if you replace the string with js/String the compiler will emit a warning about that.

So, if you want to extend your protocol to javascript primitive types, instead of using js/Number, js/String, js/Object, js/Array, js/Boolean and js/Function you should use the respective special symbols: number, string, object, array, boolean and function.

Now, it’s time to try our protocol implementation:

(invert "abc")
;; => "cba"

(invert 0)
;; => 0

(invert '(1 2 3))
;; => (3 2 1)

(invert [1 2 3])
;; => [3 2 1]

In comparison, extend-protocol does the inverse; given a protocol, it adds implementations for multiple types. This is how the syntax looks:

(extend-protocol ProtocolA
  TypeA
  (function-from-protocol-a [this]
    ;; implementation here
    )

  TypeB
  (function-from-protocol-a [this]
    ;; implementation here
    ))

Thus, the previous example could have been written equally well with this way:

(extend-protocol IInvertible
  string
  (invert [this] (apply str (reverse this)))

  cljs.core.List
  (invert [this] (reverse this))

  cljs.core.PersistentVector
  (invert [this] (into [] (reverse this))))
Participate in ClojureScript abstractions

ClojureScript itself is built up on abstractions defined as protocols. Almost all behavior in the ClojureScript language itself can be adapted to third party libraries. Let’s look at a real life example.

In previous sections, we have explained the different kinds of built-in collections. For this example we will use a set. See this snippet of code:

(def mynums #{1 2})

(filter mynums [1 2 4 5 1 3 4 5])
;; => (1 2 1)

What happened? In this case, the set type implements the ClojureScript internal IFn protocol that represents an abstraction for functions or anything callable. This way it can be used like a callable predicate in filter.

OK, but what happens if we want to use a regular expression as a predicate function for filtering a collection of strings:

(filter #"^foo" ["haha" "foobar" "baz" "foobaz"])
;; TypeError: Cannot call undefined

The exception is raised because the RegExp type does not implement the IFn protocol so it cannot behave like a callable, but that can be easily fixed:

(extend-type js/RegExp
  IFn
  (-invoke
   ([this a]
     (re-find this a))))

Let’s analyze this: we are extending the js/RegExp type so that it implements the invoke function in the IFn protocol. To invoke a regular expression a as if it were a function, call the re-find function with the object of the function and the pattern.

Now, you will be able use the regex instances as predicates in a filter operation:

(filter #"^foo" ["haha" "foobar" "baz" "foobaz"])
;; => ("foobar" "foobaz")
Introspection using Protocols

ClojureScript comes with a useful function that allows runtime introspection: satisfies?. The purpose of this function is to determine at runtime if some object (instance of some type) satisfies the concrete protocol.

So, with the previous examples, if we check if a set instance satisfies an IFn protocol, it should return true:

(satisfies? IFn #{1})
;; => true

3.11.2. Multimethods

We have previously talked about protocols which solve a very common use case of polymorphism: dispatch by type. But in some circumstances, the protocol approach can be limiting. And here, multimethods come to the rescue.

These multimethods are not limited to type dispatch only; instead, they also offer dispatch by types of multiple arguments and by value. They also allow ad-hoc hierarchies to be defined. Also, like protocols, multimethods are an "Open System", so you or any third parties can extend a multimethod for new types.

The basic constructions of multimethods are the defmulti and defmethod forms. The defmulti form is used to create the multimethod with an initial dispatch function. This is a model of what it looks like:

(defmulti say-hello
  "A polymorphic function that return a greetings message
  depending on the language key with default lang as `:en`"
  (fn [param] (:locale param))
  :default :en)

The anonymous function defined within the defmulti form is a dispatch function. It will be called in every call to the say-hello function and should return some kind of marker object that will be used for dispatch. In our example, it returns the contents of the :locale key of the first argument.

And finally, you should add implementations. That is done with the defmethod form:

(defmethod say-hello :en
  [person]
  (str "Hello " (:name person "Anonymous")))

(defmethod say-hello :es
  [person]
  (str "Hola " (:name person "Anónimo")))

So, if you execute that function over a hash map containing the :locale and optionally the :name key, the multimethod will first call the dispatch function to determine the dispatch value, then it will search for an implementation for that value. If an implementation is found, the dispatcher will execute it. Otherwise, the dispatch will search for a default implementation (if one is specified) and execute it.

(say-hello {:locale :es})
;; => "Hola Anónimo"

(say-hello {:locale :en :name "Ciri"})
;; => "Hello Ciri"

(say-hello {:locale :fr})
;; => "Hello Anonymous"

If the default implementation is not specified, an exception will be raised notifying you that some value does not have an implementation for that multimethod.

3.11.3. Hierarchies

Hierarchies are ClojureScript’s way to let you build whatever relations that your domain may require. Hierarchies are defined in term of relations between named objects, such as symbols, keywords, or types.

Hierarchies can be defined globally or locally, depending on your needs. Like multimethods, hierarchies are not limited to a single namespace. You can extend a hierarchy from any namespace, not only from the one in which it is defined.

The global namespace is more limited, for good reasons. Keywords or symbols that are not namespaced can not be used in the global hierarchy. That behavior helps prevent unexpected situations when two or more third party libraries use the same symbol for different semantics.

Defining a hierarchy

The hierarchy relations should be established using the derive function:

(derive ::circle ::shape)
(derive ::box ::shape)

We have just defined a set of relationships between namespaced keywords. In this case the ::circle is a child of ::shape, and ::box is also a child of ::shape.

Tip
The ::circle keyword syntax is a shorthand for :current.ns/circle. So if you are executing it in a REPL, ::circle will be evaluated as :cljs.user/circle.
Hierarchies and introspection

ClojureScript comes with a little toolset of functions that allows runtime introspection of globally or locally defined hierarchies. This toolset consists of three functions: isa?, ancestors, and descendants.

Let’s see an example of how it can be used with the hierarchy defined in the previous example:

(ancestors ::box)
;; => #{:cljs.user/shape}

(descendants ::shape)
;; => #{:cljs.user/circle :cljs.user/box}

(isa? ::box ::shape)
;; => true

(isa? ::rect ::shape)
;; => false
Locally defined hierarchies

As we mentioned previously, in ClojureScript you also can define local hierarchies. This can be done with the make-hierarchy function. Here is an example of how you can replicate the previous example using a local hierarchy:

(def h (-> (make-hierarchy)
           (derive :box :shape)
           (derive :circle :shape)))

Now you can use the same introspection functions with that locally defined hierarchy:

(isa? h :box :shape)
;; => true

(isa? :box :shape)
;; => false

As you can observe, in local hierarchies we can use normal (not namespace qualified) keywords, and if we execute the isa? without passing the local hierarchy parameter, it returns false as expected.

Hierarchies in multimethods

One of the big advantages of hierarchies is that they work very well together with multimethods. This is because multimethods by default use the isa? function for the last step of dispatching.

Let’s see an example to clearly understand what that means. First, we define the multimethod with the defmulti form:

(defmulti stringify-shape
  "A function that prints a human readable representation
  of a shape keyword."
  identity
  :hierarchy #'h)

With the :hierarchy keyword parameter, we indicate to the multimethod what hierarchy we want to use; if it is not specified, the global hierarchy will be used.

Second, we define an implementation for our multimethod using the defmethod form:

(defmethod stringify-shape :box
  [_]
  "A box shape")

(defmethod stringify-shape :shape
  [_]
  "A generic shape")

(defmethod stringify-shape :default
  [_]
  "Unexpected object")

Now, let’s see what happens if we execute that function with a box:

(stringify-shape :box)
;; => "A box shape"

Now everything works as expected; the multimethod executes the direct matching implementation for the given parameter. Next, let’s see what happens if we execute the same function but with the :circle keyword as the parameter which does not have the direct matching dispatch value:

(stringify-shape :circle)
;; => "A generic shape"

The multimethod automatically resolves it using the provided hierarchy, and since :circle is a descendant of :shape, the :shape implementation is executed.

Finally, if you give a keyword that isn’t part of the hierarchy, you get the :default implementation:

(stringify-shape :triangle)
;; => "Unexpected object"

3.12. Data types

Until now, we have used maps, sets, lists, and vectors to represent our data. And in most cases, this is a really great approach. But sometimes we need to define our own types, and in this book we will call them data types.

A data type provides the following:

  • A unique host-backed type, either named or anonymous.

  • The ability to implement protocols (inline).

  • Explicitly declared structure using fields or closures.

  • Map-like behavior (via records, see below).

3.12.1. Deftype

The most low-level construction in ClojureScript for creating your own types is the deftype macro. As a demonstration, we will define a type called User:

(deftype User [firstname lastname])

Once the type has been defined, we can create an instance of our User. In the following example, the . after User indicates that we are calling a constructor.

(def person (User. "Triss" "Merigold"))

Its fields can be accessed using the prefix dot notation:

(.-firstname person)
;; => "Triss"

Types defined with deftype (and defrecord, which we will see later) create a host-backed class-like object associated with the current namespace. For convenience, ClojureScript also defines a constructor function called →User that can be imported using the :require directive.

We personally do not like this type of function, and we prefer to define our own constructors with more idiomatic names:

(defn make-user
  [firstname lastname]
  (User. firstname lastname))

We use this in our code instead of →User.

3.12.2. Defrecord

The record is a slightly higher-level abstraction for defining types in ClojureScript and should be the preferred way to do it.

As we know, ClojureScript tends to use plain data types such as maps, but in most cases we need a named type to represent the entities of our application. Here come the records.

A record is a data type that implements the map protocol and therefore can be used like any other map. And since records are also proper types, they support type-based polymorphism through protocols.

In summary: with records, we have the best of both worlds, maps that can play in different abstractions.

Let’s start defining the User type but using records:

(defrecord User [firstname lastname])

It looks really similar to the deftype syntax; in fact, it uses deftype behind the scenes as a low-level primitive for defining types.

Now, look at the difference with raw types for access to its fields:

(def person (User. "Yennefer" "of Vengerberg"))

(:firstname person)
;; => "Yennefer"

(get person :firstname)
;; => "Yennefer"

As we mentioned previously, records are maps and act like them:

(map? person)
;; => true

And like maps, they support extra fields that are not initially defined:

(def person2 (assoc person :age 92))

(:age person2)
;; => 92

As we can see, the assoc function works as expected and returns a new instance of the same type but with new key-value pair. But take care with dissoc! Its behavior with records is slightly different than with maps; it will return a new record if the field being dissociated is an optional field, but it will return a plain map if you dissociate a mandatory field.

Another difference with maps is that records do not act like functions:

(def plain-person {:firstname "Yennefer", :lastname "of Vengerberg"})

(plain-person :firstname)
;; => "Yennefer"

(person :firstname)
;; => person.User does not implement IFn protocol.

For convenience, the defrecord macro, like deftype, exposes a →User function, as well as an additional map→User constructor function. We have the same opinion about that constructor as with deftype defined ones: we recommend defining your own instead of using the other ones. But as they exist, let’s see how they can be used:

(def cirilla (->User "Cirilla" "Fiona"))
(def yen (map->User {:firstname "Yennefer"
                     :lastname "of Vengerberg"}))

3.12.3. Implementing protocols

Both type definition primitives that we have seen so far allow inline implementations for protocols (explained in a previous section). Let’s define one for example purposes:

(defprotocol IUser
  "A common abstraction for working with user types."
  (full-name [_] "Get the full name of the user."))

Now, you can define a type with inline implementation for an abstraction, in our case the IUser:

(defrecord User [firstname lastname]
  IUser
  (full-name [_]
    (str firstname " " lastname)))

;; Create an instance.
(def user (User. "Yennefer" "of Vengerberg"))

(full-name user)
;; => "Yennefer of Vengerberg"

3.12.4. Reify

The reify macro is an ad hoc constructor you can use to create objects without pre-defining a type. Protocol implementations are supplied the same as deftype and defrecord, but in contrast, reify does not have accessible fields.

This is how we can emulate an instance of the user type that plays well with the IUser abstraction:

(defn user
  [firstname lastname]
  (reify
    IUser
    (full-name [_]
      (str firstname " " lastname))))

(def yen (user "Yennefer" "of Vengerberg"))
(full-name yen)
;; => "Yennefer of Vengerberg"

3.12.5. Specify

specify! is an advanced alternative to reify, allowing you to add protocol implementations to an existing JavaScript object. This can be useful if you want to graft protocols onto a JavaScript library’s components.

(def obj #js {})

(specify! obj
  IUser
  (full-name [_]
    "my full name"))

(full-name obj)
;; => "my full name"

specify is an immutable version of specify! that can be used on immutable, copyable values implementing ICloneable (e.g. ClojureScript collections).

(def a {})

(def b (specify a
         IUser
         (full-name [_]
           "my full name")))

(full-name a)
;; Error: No protocol method IUser.full-name defined for type cljs.core/PersistentArrayMap: {}

(full-name b)
;; => "my full name"

3.13. Host interoperability

ClojureScript, in the same way as its brother Clojure, is designed to be a "guest" language. This means that the design of the language works well on top of an existing ecosystem such as JavaScript for ClojureScript and the JVM for Clojure.

3.13.1. The types

ClojureScript, unlike what you might expect, tries to take advantage of every type that the platform provides. This is a (perhaps incomplete) list of things that ClojureScript inherits and reuses from the underlying platform:

  • ClojureScript strings are JavaScript Strings.

  • ClojureScript numbers are JavaScript Numbers.

  • ClojureScript nil is a JavaScript null.

  • ClojureScript regular expressions are JavaScript RegExp instances.

  • ClojureScript is not interpreted; it is always compiled down to JavaScript.

  • ClojureScript allows easy call to platform APIs with the same semantics.

  • ClojureScript data types internally compile to objects in JavaScript.

On top of it, ClojureScript builds its own abstractions and types that do not exist in the platform, such as Vectors, Maps, Sets, and others that are explained in preceding sections of this chapter.

3.13.2. Interacting with platform types

ClojureScript comes with a little set of special forms that allows it to interact with platform types such as calling object methods, creating new instances, and accessing object properties.

Access to the platform

ClojureScript has a special syntax to access the entire platform environment through the js/ special namespace. This is an example of an expression to execute JavaScript’s built-in parseInt function:

(js/parseInt "222")
;; => 222
Creating new instances

ClojureScript has two ways to create instances:

Using the new special form

(new js/RegExp "^foo$")

Using the . special form

(js/RegExp. "^foo$")

The last one is the recommended way to create instances. We are not aware of any real differences between the two forms, but in the ClojureScript community, the last one is used most often.

Invoke instance methods

To invoke methods of some object instance, as opposed to how it is done in JavaScript (e.g., obj.method(), the method name comes first like any other standard function in Lisp languages but with a little variation: the function name starts with special form ..

Let’s see how we can call the .test() method of a regexp instance:

(def re (js/RegExp "^Clojure"))

(.test re "ClojureScript")
;; => true

You can invoke instance methods on JavaScript objects. The first example follows the pattern you have seen; the last one is a shortcut:

(.sqrt js/Math 2)
;; => 1.4142135623730951
(js/Math.sqrt 2)
;; => 1.4142135623730951
Access to object properties

Access to an object’s properties is really very similar to calling a method. The difference is that instead of using the . you use .-. Let’s see an example:

(.-multiline re)
;; => false
(.-PI js/Math)
;; => 3.141592653589793
Property access shorthand

Symbols with the js/ prefix can contain dots to denote nested property access. Both of the following expressions invoke the same function:

(.log js/console "Hello World")

(js/console.log "Hello World")

And both of the following expressions access the same property:

(.-PI js/Math)
;; => 3.141592653589793

js/Math.PI
;; => 3.141592653589793
JavaScript objects

ClojureScript has different ways to create plain JavaScript objects; each one has its own purpose. The basic one is the js-obj function. It accepts a variable number of pairs of keys and values and returns a JavaScript object:

(js-obj "country" "FR")
;; => #js {:country "FR"}

The return value can be passed to some kind of third party library that accepts a plain JavaScript object, but you can observe the real representation of the return value of this function. It is really another form for doing the same thing.

Using the reader macro #js consists of prepending it to a ClojureScript map or vector, and the result will be transformed to plain JavaScript:

(def myobj #js {:country "FR"})

The translation of that to plain JavaScript is similar to this:

var myobj = {country: "FR"};

As explained in the previous section, you can also access the plain object properties using the .- syntax:

(.-country myobj)
;; => "FR"

And as JavaScript objects are mutable, you can set a new value for some property using the set! function:

(set! (.-country myobj) "KR")
Conversions

The inconvenience of the previously explained forms is that they do not make recursive transformations, so if you have nested objects, the nested objects will not be converted. Consider this example that uses Clojurescript maps, then a similar one with JavaScript objects:

(def clj-map {:country {:code "FR" :name "France"}})
;; => {:country {:code "FR", :name "France"}}
(:code (:country clj-map))
;; => "FR"

(def js-obj #js {:country {:code "FR" :name "France"}})
;; => #js {:country {:code "FR", :name "France"}
(.-country js-obj)
;; => {:code "FR", :name "France"}
(.-code (.-country js-obj))
;; => nil

To solve that use case, ClojureScript comes with the clj→js and js→clj functions that transform Clojure collection types into JavaScript and back. Note that the conversion to ClojureScript changes the :country keyword to a string.

(clj->js {:foo {:bar "baz"}})
;; => #js {:foo #js {:bar "baz"}}
(js->clj #js {:country {:code "FR" :name "France"}})
;; => {"country" {:code "FR", :name "France"}}

In the case of arrays, there is a specialized function into-array that behaves as expected:

(into-array ["France" "Korea" "Peru"])
;; => #js ["France" "Korea" "Peru"]

Keep in mind that clj→js is not the canonical way to transform data from clojurescript to js, that function is for debugging, because it is not exactly the most efficient way to do this operation. It is recommended that you build the transformation data structures specific to your domain rather than using the generic clj→js.

Arrays

In the previous example, we saw how we can create an array from an existing ClojureScript collection. But there is another function for creating arrays: make-array.

Creating a preallocated array with length 10
(def a (make-array 10))
;; => #js [nil nil nil nil nil nil nil nil nil nil]

In ClojureScript, arrays also play well with sequence abstractions, so you can iterate over them or simply get the number of elements with the count function:

(count a)
;; => 10

As arrays in the JavaScript platform are a mutable collection type, you can access a concrete index and set the value at that position:

(aset a 0 2)
;; => 2
a
;; => #js [2 nil nil nil nil nil nil nil nil nil]

Or access in an indexed way to get its values:

(aget a 0)
;; => 2

In JavaScript, array index access is equivalent to object property access, so you can use the same functions for interacting with plain objects:

(def b #js {:hour 16})
;; => #js {:hour 16}

(aget b "hour")
;; => 16

(aset b "minute" 22)
;; => 22

b
;; => #js {:hour 16, :minute 22}

3.14. State management

We’ve learned that one of ClojureScript’s fundamental ideas is immutability. Both scalar values and collections are immutable in ClojureScript, except those mutable types present in the JS host like Date.

Immutability has many great properties but we are sometimes faced with the need to model values that change over time. How can we achieve this if we can’t change data structures in place?

3.14.1. Vars

Vars can be redefined at will inside a namespace but there is no way to know when they change. The inability to redefine vars from other namespaces is a bit limiting; also, if we are modifying state, we’re probably interested in knowing when it occurs.

3.14.2. Atoms

ClojureScript gives us the Atom type, which is an object containing a value that can be altered at will. Besides altering its value, it also supports observation through watcher functions that can be attached and detached from it and validation for ensuring that the value contained in the atom is always valid.

If we were to model an identity corresponding to a person called Ciri, we could wrap an immutable value containing Ciri’s data in an atom. Note that we can get the atom’s value with the deref function or using its shorthand @ notation:

(def ciri (atom {:name "Cirilla" :lastname "Fiona" :age 20}))
;; #<Atom: {:name "Cirilla", :lastname "Fiona", :age 20}>

(deref ciri)
;; {:name "Cirilla", :lastname "Fiona", :age 20}

@ciri
;; {:name "Cirilla", :lastname "Fiona", :age 20}

We can use the swap! function on an atom to alter its value with a function. Since Ciri’s birthday is today, let’s increment her age count:

(swap! ciri update :age inc)
;; {:name "Cirilla", :lastname "Fiona", :age 21}

@ciri
;; {:name "Cirilla", :lastname "Fiona", :age 21}

The reset! functions replaces the value contained in the atom with a new one:

(reset! ciri {:name "Cirilla", :lastname "Fiona", :age 22})
;; {:name "Cirilla", :lastname "Fiona", :age 22}

@ciri
;; {:name "Cirilla", :lastname "Fiona", :age 22}
Observation

We can add and remove watcher functions for atoms. Whenever the atom’s value is changed through a swap! or reset!, all the atom’s watcher functions will be called. Watchers are added with the add-watch function. Notice that each watcher has a key associated (:logger in the example) to it which is later used to remove the watch from the atom.

(def a (atom))

(add-watch a :logger (fn [key the-atom old-value new-value]
                       (println "Key:" key "Old:" old-value "New:" new-value)))

(reset! a 42)
;; Key: :logger Old: nil New: 42
;; => 42

(swap! a inc)
;; Key: :logger Old: 42 New: 43
;; => 43

(remove-watch a :logger)

3.14.3. Volatiles

Volatiles, like atoms, are objects containing a value that can be altered. However, they don’t provide the observation and validation capabilities that atoms provide. This makes them slightly more performant and a more suitable mutable container to use inside stateful functions that don’t need observation nor validation.

Their API closely resembles that of atoms. They can be dereferenced to grab the value they contain and support swapping and resetting with vswap! and vreset! respectively:

(def ciri (volatile! {:name "Cirilla" :lastname "Fiona" :age 20}))
;; #<Volatile: {:name "Cirilla", :lastname "Fiona", :age 20}>

(volatile? ciri)
;; => true

(deref ciri)
;; {:name "Cirilla", :lastname "Fiona", :age 20}

(vswap! ciri update :age inc)
;; {:name "Cirilla", :lastname "Fiona", :age 21}

(vreset! ciri {:name "Cirilla", :lastname "Fiona", :age 22})
;; {:name "Cirilla", :lastname "Fiona", :age 22}

Note that another difference with atoms is that the constructor of volatiles uses a bang at the end. You create volatiles with volatile! and atoms with atom.

4. Tooling & Compiler

This chapter will cover a little introduction to existing tooling for making things easy when developing using ClojureScript. Unlike the previous chapter, this chapter intends to tell different stories each independent of the other.

4.1. Getting Started with the Compiler

At this point, you are surely very bored with the constant theoretical explanations about the language itself and will want to write and execute some code. The goal of this section is to provide a little practical introduction to the ClojureScript compiler.

The ClojureScript compiler takes the source code that has been split over numerous directories and namespaces and compiles it down to JavaScript. Today, JavaScript has a great number of different environments where it can be executed - each with its own peculiarities.

For this case we are going to use shadow-cljs as a frontend for the compiler. It is possible to use the compiler directly or use other frontends, but in this case we will focus on just one, which in my opinion best resolves usability.

4.1.1. Execution environments

What is an execution environment? An execution environment is an engine where JavaScript can be executed. For example, the most popular execution environment is a browser (Chrome, Firefox, …​) followed by the second most popular - nodejs.

There are others, such as Rhino (JDK 6+), Nashorn (JDK 8+), QtQuick (QT),…​ but none of them have significant differences from the first two. So, ClojureScript at the moment may compile code to run in the browser or in nodejs-like environments out of the box.

4.1.2. Setup for NodeJS

This chapter supposes you have a properly installed nodejs (v20.10.0) and JVM (JDK21). Other versions probably work but all this is tested under specified versions.

It also will be nice to have the rlwrap tool, which can be installed this way on a debian like linux distributions:

sudo apt-get install rlwrap

So, lets create our directory stucture and the first package.json file:

mkdir -p mynodeapp/src/mynodeapp
touch mynodeapp/package.json
touch mynodeapp/shadow-cljs.edn
touch mynodeapp/src/mynodeapp/main.cljs

Then, lets define our package.json content:

{
  "name": "mynodeapp",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "server": "shadow-cljs server",
    "watch": "shadow-cljs watch app",
    "compile": "shadow-cljs compile app"
  },
  "author": "",
  "license": "MPL-2.0",
  "devDependencies": {
    "shadow-cljs": "^2.26.2"
  }
}

And execute the npm install for correctly install shadow-cljs dependency

Then, setup the basic shadow-cljs.edn confguration:

{:dependencies
 []

 :source-paths
 ["src"]

 :builds
 {:app {:target :node-script
        :output-to "target/app.js"
        :main mynodeapp.main/main}}}

And finally, put the following content on the mynodeapp/main.cljs file:

(ns mynodeapp.main)

(defn main
  [& args]
  (println "Hello world!"))
Note
It is very important that the declared namespace in the file exactly matches the directory structure. This is the way ClojureScript structures its source code.

Now it’s time to compile the project:

$ npm run compile

> mynodeapp@1.0.0 compile
> shadow-cljs compile app

shadow-cljs - config: /home/user/playground/mynodeapp/shadow-cljs.edn
[:app] Compiling ...
[:app] Build completed. (45 files, 1 compiled, 0 warnings, 1.65s)

And when it finishes, execute the result using node binary:

$ node target/main.js
Hello world!

The shadow-cljs guide is very extensive, and you can read more detailed documentation here: https://shadow-cljs.github.io/docs/UsersGuide.html#target-node

4.1.3. Setup for Browser

In this section we are going to create an application similar to the "hello world" example from the previous section to run in the browser environment. The minimal requirement for this application is just a browser that can execute JavaScript.

The process is almost the same, and the directory structure is very similar, with few additions.

mkdir -p mywebapp/public
mkdir -p mywebapp/src/mywebapp
touch mywebapp/public/index.html
touch mywebapp/src/mywebapp/main.cljs
touch mywebapp/package.json
touch mywebapp/shadow-cljs.edn

Set setup the package.json file:

{
  "name": "mywebapp",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "server": "shadow-cljs server",
    "watch": "shadow-cljs watch app",
    "compile": "shadow-cljs compile app"
  },
  "author": "",
  "license": "MPL-2.0",
  "devDependencies": {
    "shadow-cljs": "^2.26.2"
  }
}

Execute the npm install for correctly install the shadow-cljs dependency.

Then, setup shadow-cljs configuration on shadow-cljs.edn file:

{:dev-http {8888 "public"}

 :dependencies
 []

 :source-paths
 ["src"]

 :builds
 {:app {:target :browser
        :output-dir "public/js"
        :asset-path "/js"
        :modules {:main {:entries [mywebapp.main]
                         :init-fn mywebapp.main/main}}}}}

Write new content to the src/mywebapp/main.cljs file:

(ns mywebapp.core)

(defn main
  []
  (println "Hello world!"))

Once all this is ready, instead of just compiling it, we start a watch process:

$ npm run watch

> mywebapp@1.0.0 watch
> shadow-cljs watch app

shadow-cljs - config: /home/user/playground/mywebapp/shadow-cljs.edn
shadow-cljs - HTTP server available at http://localhost:8888
shadow-cljs - server version: 2.26.2 running at http://localhost:9630
shadow-cljs - nREPL server started on port 44159
shadow-cljs - watching build :app
[:app] Configuring build.
[:app] Compiling ...
[:app] Build completed. (119 files, 0 compiled, 0 warnings, 2.58s)

The watch also starts a development http server at http://localhost:8888 for access our brand new web application. But we still need a last step: we need to create an index.html where we’re going to load our recently compiled js.

On the file public/index.html
<!DOCTYPE html>
<html>
  <header>
    <meta charset="utf-8" />
    <title>Hello World from ClojureScript</title>
  </header>
  <body>
    <script src="js/main.js"></script>
  </body>
</html>

So, open the browser on http://localhost:8888 and open the devconsole with F12 key and see the Hello World printed on the console tab.

The shadow-cljs guide is very extensive, and you can read more detailed documentation here: https://shadow-cljs.github.io/docs/UsersGuide.html#target-browser

4.1.4. Working with the REPL

Although you can create a source file and compile it every time you want to try something out in ClojureScript, it’s easier to use the REPL. REPL stands for:

  • Read - get input from the keyboard

  • Evaluate the input

  • Print the result

  • Loop back for more input

In other words, the REPL lets you try out ClojureScript concepts and get immediate feedback.

ClojureScript comes with support for executing the REPL in different execution environments, each of which has its own advantages and disadvantages. For example, you can run a REPL in nodejs but in that environment you don’t have any access to the DOM. Which REPL environment is best for you depends on your specific needs and requirements.

Node REPL
$ shadow-cljs node-repl

This starts a blank CLJS REPL with an already connected node process.

Important
If you exit the Node REPL the node process is also killed!

The node-repl lets you get started without any additional configuration. It has access to all your code via the usual means, ie. (require '[your.core :as x]). Since it is not connected to any build it does not do any automatic rebuilding of code when your files change and does not provide hot-reload.

Browser REPL
$ shadow-cljs browser-repl

This starts a blank CLJS REPL and will open an associated Browser window where the code will execute. Besides running in the Browser this has all the same functionality as the above node-repl.

Important
If you close the Browser window the REPL will stop working.

4.1.5. Build-specific REPL

The node-repl and browser-repl work without any specific build configuration. That means they will only do whatever you tell them to do but nothing on their own.

But there is also an option to connect to a build specific repl, for which to work correctly you need 2 things:

  • a running watch for your build

  • connect the JS runtime of the :target. Meaning if you are using the :browser target you need to open a Browser that has the generated JS loaded. For node.js builds that means running the node process.

Once you have both you can connect to the CLJS REPL via the command line or from the Clojure REPL.

# One terminal
$ npx shadow-cljs watch app
...

# different terminal
$ npx shadow-cljs cljs-repl app
shadow-cljs - connected to server
[3:1]~cljs.user=>
REPL
Tip
type :repl/quit to exit the REPL. This will only exit the REPL, the watch will remain running. TIP: You may run multiple watch "workers" in parallel and connect/disconnect to their REPLs at any given time.

4.2. The Closure Library

The Google Closure Library is a javascript library developed by Google. It has a modular architecture, and provides cross-browser functions for DOM manipulations and events, ajax and JSON, and other features.

The Google Closure Library is written specifically to take advantage of the Closure Compiler (which is used internally by the ClojureScript compiler).

ClojureScript is built on the Google Closure Compiler and Closure Library. In fact, ClojureScript namespaces are Closure modules. This means that you can interact with the Closure Library very easily:

(ns yourapp.core
  (:require [goog.dom :as dom]))

(def element (dom/getElement "body"))

This code snippet shows how you can import the dom module of the Closure library and use a function declared in that module.

Additionally, the closure library exposes "special" modules that behave like a class or object. To use these features, you must use the :import directive in the (ns …​) form:

(ns yourapp.core
  (:import goog.History))

(def instance (History.))

In a Clojure program, the :import directive is used for host (Java) interop to import Java classes. If, however, you define types (classes) in ClojureScript, you should use the standard :require directive and not the :import directive.

You can found the reference to all namespaces in the closure library here: http://google.github.io/closure-library/api/

To properly understand how we can use the "batteries included" of google closure library, lets add some functionality to our mywebapp example application.

Lets update our public/index.html with the following content:

<!DOCTYPE html>
<html>
  <head>
    <title>leapyears</title>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  </head>
  <body>
    <section class="viewport">
      <div id="result">
        ----
      </div>

      <form action="" method="">
        <label for="year">Input a year</label>
        <input id="year" name="year" />
      </form>
    </section>

    <script src="/js/main.js"></script>
  </body>
</html>

And then, update the src/mywebapp/main.cljs file with:

(ns mywebapp.main
  (:require
   [goog.dom :as dom]
   [goog.events :as events]
   [cljs.reader :refer [read-string]]))

(defn leap?
  [year]
  (or (zero? (js-mod year 400))
      (and (pos? (js-mod year 100))
           (zero? (js-mod year 4)))))

(defn on-change
  [event]
  (let [target (.-target event)
        result (dom/getElement "result")
        value (read-string (.-value target))]
    (if (leap? value)
      (set! (.-innerHTML result) "YES")
      (set! (.-innerHTML result) "NO"))))

(defn main
  []
  (let [input (dom/getElement "year")]
    (events/listen input "keyup" on-change)))

Now, we can compile the project in the same way as previously

npm run watch

Finally, open the http://locahost:8888 in a browser. Typing a year in the textbox should display an indication of its leap year status.

4.3. Dependency management

Until now, we have used the builtin Clojure(Script) toolchain to compile our source files to JavaScript. Now this is a time to understand how manage external and/or third party dependencies. Lets use our mywebapp to add some functionality with help of an external dependencies.

With shadow-cljs we have the following approaches to add dependencies:

  • Adding a native/cljs dependency

  • Adding a npm dependency

  • Adding a global dependency

4.3.1. Adding native dependencies

In this example we will use the Cuerdas (a string manipulation library build especifically for Clojure(Script)) for improve the previous functionality of the mywebapp.

Let’s update our shadow-cljs.edn file with the dependency:

{:dependencies
 [[funcool/cuerdas "2022.06.16-403"]]

 ;; [...]
 }

And the following modifications to the src/mywebapp/main.cljs file:

(ns mywebapp.main
  (:require
    [goog.dom :as dom]
    [goog.events :as events]
    [cuerdas.core :as str]
    [cljs.reader :refer [read-string]]))

;; [...]

(defn on-change
  [event]
  (let [target (.-target event)
        value (read-string (.-value target))]

    (if (str/blank? value)
      (set! (.-innerHTML result) "---")
      (if (leap? value)
        (set! (.-innerHTML result) "YES")
        (set! (.-innerHTML result) "NO")))))

Now, if you run the build or watch command, the new declared dependency will be downloaded and the application will be compiled with this dependency included.

4.3.2. NPM Dependencies

On the other hand, not all dependencies are available as native packages. But one of the advantages of ClojureScript is that it embraces being able to integrate with the host language, in this case JS. For this case, shadow-cljs integrates quite well with the NPM ecosystem.

Lets add the date-fns NPM dependency:

$ npm add date-fns

And then, lets modify our code to use that library:

(ns mywebapp.main
  (:require
   ["date-fns" :as df]
   [goog.dom :as dom]
   [cuerdas.core :as str]
   [goog.events :as events]))

(defn on-change
  [event]
  (let [result (dom/getElement "result")
        target (.-target event)
        value  (.-value target)]

    (if (str/blank? value)
      (set! (.-innerHTML result) "---")
      (if (df/isLeapYear value)
        (set! (.-innerHTML result) "YES")
        (set! (.-innerHTML result) "NO")))))

(defn main
  []
  (let [input (dom/getElement "year")]
    (events/listen input "keyup" on-change)))

You can observe that, NPM depenencies are declared as strings on the ns section. Then, you just use the isLeapYear exported function from the date-fns library like any other function.

You can read the whole detail on how it works here: https://shadow-cljs.github.io/docs/UsersGuide.html#npm

4.3.3. Glonal dependency

There’s not much to really explain here, in this case, include whatever library is needed from the CDN or elsewhere in the index.html and access the global export that that library uses using clojurescript’s js/ syntax to access globally defined things. Just as you would do, for example, to access the location object.

Example:

(js/console.log (.-href js/location))

4.4. Dealing with JS files

There are also circumstances in which it is better to use the host’s native language to implement some feature or you already have a piece of code implementing some algorithm and you want to be able to use it from CLJS.

For this case, we have two options: an ESM module or a Closure module.

4.4.1. Closure compatible module

Lets start with a Closure module definition.

The main advantage of this approach is that there are no distinction between a clojurescript namespace and a file defined using closure module style. You import it in the same way independently on how it is implemented. Also if you have a library that is written to be compatible with closure module system and you want to include it on your project. And finally, it is the standard way how clojurescript compiler generates code from your .cljs files, so it works with or without shadow-cljs tooling.

Obviously, the main disadvantage is that is uses a custom, not very common approach for defining it.

So, for understand it correctly, lets use our mywebapp application and add an util module. This is the most simplest case, because closure modules are directly compatible with cljs and you can mix your clojurescript code with javascript code without any additional steps.

touch src/mywebapp/util.js

And then, we implement the isLeapYear in javascript:

src/mywebapp/util.js
goog.provide("mywebapp.util");

goog.scope(function() {
  let module = mywebapp.util;

  module.isLeapYear = function(val) {
    val = parseInt(val, 10);
    return (0 === (val % 400)) || (((val % 100) > 0) && (0 === (val % 4)));
  };
});

Then modify the src/mywebapp/main.cljs file to use our util module:

(ns mywebapp.main
  (:require
   [mywebapp.util :as util]
   [goog.dom :as dom]
   [cuerdas.core :as str]
   [goog.events :as events]))

(defn on-change
  [event]
  (let [result (dom/getElement "result")
        target (.-target event)
        value  (.-value target)]

    (if (str/blank? value)
      (set! (.-innerHTML result) "---")
      (if (util/isLeapYear value)
        (set! (.-innerHTML result) "YES")
        (set! (.-innerHTML result) "NO")))))

(defn main
  []
  (let [input (dom/getElement "year")]
    (events/listen input "keyup" on-change)))

As you can observe, the require entry is indistinguible from any other, because it integrates 100% with the cljs build process.

4.4.2. ESM Modules

This is other way to define an companion modules using JS language; is the recommended way to do it on shadow-cljs and the main advantage of it is the familiar ESM syntax.

So, lets proceed to define/overwrite the same module used in previous example to use the ESM syntax.

export function isLeapYear (val) {
  val = parseInt(val, 10);
  return (0 === (val % 400)) || (((val % 100) > 0) && (0 === (val % 4)));
}

Then, change the require on the src/mywebapp/main.cljs file to:

(ns mywebapp.main
  (:require
   ["./util.js" :as util]
   [goog.dom :as dom]
   [cuerdas.core :as str]
   [goog.events :as events]))

;; [...]

You can read a more detailed information about this here: https://shadow-cljs.github.io/docs/UsersGuide.html#classpath-js

4.5. Unit testing

As you might expect, testing in ClojureScript consists of the same concepts widely used by other language such as Clojure, Java, Python, JavaScript, etc.

Regardless of the language, the main objective of unit testing is to run some test cases, verifying that the code under test behaves as expected and returns without raising unexpected exceptions.

The immutablity of ClojureScript data structures helps to make programs less error prone, and facilitates testing a little bit. Another advantage of ClojureScript is that it tends to use plain data instead of complex objects. Building "mock" objects for testing is thus greatly simplified.

4.5.1. Setup first Test

The "official" ClojureScript testing framework is in the "cljs.test" namespace. It is a very simple library, but it should be more than enough for our purposes.

There are other libraries that offer additional features or directly different approaches to testing, such as test.check. However, we will not cover them here.

We will reuse the mywebapp project structure and we will add testing to it. Let’s create the test file src/mywebapp/util_test.cljs:

(ns mywebapp.util-test
  (:require
   [cljs.test :as t]
   ["./util.js" :as util]))

(t/deftest is-leap
  (t/is (true? (util/isLeapYear "2024"))))

Now, lets add a new build target to our shadow-cljs.edn configuration file:

{:dev-http {8888 "public"}

 :dependencies
 [[funcool/cuerdas "2022.06.16-403"]]

 :source-paths
 ["src" "test"]

 :builds
 {:app
  {:target :browser
   :output-dir "public/js"
   :asset-path "/js"
   :modules {:main {:entries [mywebapp.main]
                    :init-fn mywebapp.main/main}}}
  :test
  {:target    :node-test
   :output-to "target/test.js"
   :ns-regexp "-test$"
   :autorun   true}}}

Now, lets comple it and run tests:

~~~~
$ npx shadow-cljs watch test
shadow-cljs - config: /home/user/playground/mywebapp/shadow-cljs.edn
shadow-cljs - HTTP server available at http://localhost:8888
shadow-cljs - server version: 2.26.2 running at http://localhost:9630
shadow-cljs - nREPL server started on port 40217
shadow-cljs - watching build :test
[:test] Configuring build.
[:test] Compiling ...

Testing mywebapp.util-test

Ran 1 tests containing 1 assertions. 0 failures, 0 errors.

[:test] Build completed. (52 files, 2 compiled, 0 warnings, 2.38s) ~~

It will recompile and rerun the tests on each code change.

4.5.2. Async Testing

One of the peculiarities of ClojureScript is that it runs in an asynchronous, single-threaded execution environment, which has its challenges.

In an async execution environment, we should be able to test asynchronous functions. To this end, the ClojureScript testing library offers the async macro, allowing you to create tests that play well with asynchronous code.

First, we need to write a function that works in an asynchronous way. For this purpose, we will create the async-leap? predicate that will do the same operation but asychronously return a result using a callback:

(defn async-leap?
  [year callback]
  (js/setImmediate #(callback (util/isLeap year))))

The JavaScript function setImmediate is used to emulate an asynchronous task, and the callback is executed with the result of that predicate.

To test it, we should write a test case using the previously mentioned async macro:

(t/deftest my-async-test
  (t/async done
    (async-leap? 1980 (fn [result]
                        (t/is (true? result))
                              (done)))))

The done function exposed by the async macro should be called after the asynchronous operation is finished and all assertions have run.

It is very important to execute the done function only once. Omitting it or executing it twice may cause strange behavior and should be avoided.

5. Language (advanced topics)

This chapter intends to explain some advanced topics that are part of the language and that does not fit in the first chapter. The good candidates for this section are transducers, core protocols, transients, metadata. In summary: topics that are not mandatory for understand the language.

5.2. Reader Conditionals

This language feature allows different dialects of Clojure to share common code that is mostly platform independent but need some platform dependent code.

To use reader conditionals, all you need is to rename your source file with .cljs extension to one with .cljc, because reader conditionals only work if they are placed in files with .cljc extension.

5.2.1. Standard (#?)

There are two types of reader conditionals, standard and splicing. The standard reader conditional behaves similarly to a traditional cond and the syntax looks like this:

(defn parse-int
  [v]
  #?(:clj  (Integer/parseInt v)
     :cljs (js/parseInt v)))

As you can observe, #? reading macro looks very similar to cond, the difference is that the condition is just a keyword that identifies the platform, where :cljs is for ClojureScript and :clj is for Clojure. The advantage of this approach, is that it is evaluated at compile time so no runtime performance overhead introduced for using this.

5.2.2. Splicing (#?@)

The splicing reader conditional works in the same way as the standard but also allows splicing lists into the containing form. The #?@ reader macro is used for that and the code looks like this:

(defn make-list
  []
  (list #?@(:clj  [5 6 7 8]
            :cljs [1 2 3 4])))

;; On ClojureScript
(make-list)
;; => (1 2 3 4)

The ClojureScript compiler will read that code as this:

(defn make-list
  []
  (list 1 2 3 4))

The splicing reader conditional can’t be used to splice multiple top level forms, so the following code is illegal:

#?@(:cljs [(defn func-a [] :a)
           (defn func-b [] :b)])
;; => #error "Reader conditional splicing not allowed at the top level."

If you need so, you can use multiple forms or just use do block to group multiple forms together:

#?(:cljs (defn func-a [] :a))
#?(:cljs (defn func-b [] :b))

;; Or

#?(:cljs
   (do
     (defn func-a [] :a)
     (defn func-b [] :b)))

5.2.4. Data transformation

ClojureScript offers a rich vocabulary for data transformation in terms of the sequence abstraction, which makes such transformations highly general and composable. Let’s see how we can combine several collection processing functions to build new ones. We will be using a simple problem throughout this section: splitting grape clusters, filtering out the rotten ones, and cleaning them. We have a collection of grape clusters like the following:

(def grape-clusters
  [{:grapes [{:rotten? false :clean? false}
             {:rotten? true :clean? false}]
    :color :green}
   {:grapes [{:rotten? true :clean? false}
             {:rotten? false :clean? false}]
    :color :black}])

We are interested in splitting the grape clusters into individual grapes, discarding the rotten ones and cleaning the remaining grapes so they are ready for eating. We are well-equipped in ClojureScript for this data transformation task; we could implement it using the familiar map, filter and mapcat functions:

(defn split-cluster
  [c]
  (:grapes c))

(defn not-rotten
  [g]
  (not (:rotten? g)))

(defn clean-grape
  [g]
  (assoc g :clean? true))

(->> grape-clusters
     (mapcat split-cluster)
     (filter not-rotten)
     (map clean-grape))
;; => ({rotten? false :clean? true} {:rotten? false :clean? true})

In the above example we succintly solved the problem of selecting and cleaning the grapes, and we can even abstract such transformations by combining the mapcat, filter and map operations using partial application and function composition:

(def process-clusters
  (comp
    (partial map clean-grape)
    (partial filter not-rotten)
    (partial mapcat split-cluster)))

(process-clusters grape-clusters)
;; => ({rotten? false :clean? true} {:rotten? false :clean? true})

The code is very clean, but it has a few problems. For example, each call to map, filter and mapcat consumes and produces a sequence that, although lazy, generates intermediate results that will be discarded. Each sequence is fed to the next step, which also returns a sequence. Wouldn’t be great if we could do the transformation in a single transversal of the grape-cluster collection?

Another problem is that even though our process-clusters function works with any sequence, we can’t reuse it with anything that is not a sequence. Imagine that instead of having the grape cluster collection available in memory it is being pushed to us asynchronously in a stream. In that situation we couldn’t reuse process-clusters since usually map, filter and mapcat have concrete implementations depending on the type.

5.2.5. Generalizing to process transformations

The process of mapping, filtering or mapcatting isn’t necessarily tied to a concrete type, but we keep reimplementing them for different types. Let’s see how we can generalize such processes to be context independent. We’ll start by implementing naive versions of map and filter first to see how they work internally:

(defn my-map
  [f coll]
  (when-let [s (seq coll)]
    (cons (f (first s)) (my-map f (rest s)))))

(my-map inc [0 1 2])
;; => (1 2 3)

(defn my-filter
  [pred coll]
  (when-let [s (seq coll)]
    (let [f (first s)
          r (rest s)]
      (if (pred f)
        (cons f (my-filter pred r))
        (my-filter pred r)))))

(my-filter odd? [0 1 2])
;; => (1)

As we can see, they both assume that they receive a seqable and return a sequence. Like many recursive functions they can be implemented in terms of the already familiar reduce function. Note that functions that are given to reduce receive an accumulator and an input and return the next accumulator. We’ll call these types of functions reducing functions from now on.

(defn my-mapr
  [f coll]
  (reduce (fn [acc input]         ;; reducing function
            (conj acc (f input)))
          []                      ;; initial value
          coll))                  ;; collection to reduce

(my-mapr inc [0 1 2])
;; => [1 2 3]

(defn my-filterr
  [pred coll]
  (reduce (fn [acc input]         ;; reducing function
            (if (pred input)
              (conj acc input)
              acc))
          []                      ;; initial value
          coll))                  ;; collection to reduce

(my-filterr odd? [0 1 2])
;; => [1]

We’ve made the previous versions more general since using reduce makes our functions work on any thing that is reducible, not just sequences. However you may have noticed that, even though my-mapr and my-filterr don’t know anything about the source (coll) they are still tied to the output they produce (a vector) both with the initial value of the reduce ([]) and the hardcoded conj operation in the body of the reducing function. We could have accumulated results in another data structure, for example a lazy sequence, but we’d have to rewrite the functions in order to do so.

How can we make these functions truly generic? They shouldn’t know about either the source of inputs they are transforming nor the output that is generated. Have you noticed that conj is just another reducing function? It takes an accumulator and an input and returns another accumulator. So, if we parameterise the reducing function that my-mapr and my-filterr use, they won’t know anything about the type of the result they are building. Let’s give it a shot:

(defn my-mapt
  [f]                         ;; function to map over inputs
  (fn [rfn]                   ;; parameterised reducing function
    (fn [acc input]           ;; transformed reducing function, now it maps `f`!
      (rfn acc (f input)))))

(def incer (my-mapt inc))

(reduce (incer conj) [] [0 1 2])
;; => [1 2 3]

(defn my-filtert
  [pred]                      ;; predicate to filter out inputs
  (fn [rfn]                   ;; parameterised reducing function
    (fn [acc input]           ;; transformed reducing function, now it discards values based on `pred`!
      (if (pred input)
        (rfn acc input)
        acc))))

(def only-odds (my-filtert odd?))

(reduce (only-odds conj) [] [0 1 2])
;; => [1]

That’s a lot of higher-order functions so let’s break it down for a better understanding of what’s going on. We’ll examine how my-mapt works step by step. The mechanics are similar for my-filtert, so we’ll leave it out for now.

First of all, my-mapt takes a mapping function; in the example we are giving it inc and getting another function back. Let’s replace f with inc to see what we are building:

(def incer (my-mapt inc))
;; (fn [rfn]
;;   (fn [acc input]
;;     (rfn acc (inc input))))
;;               ^^^

The resulting function is still parameterised to receive a reducing function to which it will delegate, let’s see what happens when we call it with conj:

(incer conj)
;; (fn [acc input]
;;   (conj acc (inc input)))
;;    ^^^^

We get back a reducing function which uses inc to transform the inputs and the conj reducing function to accumulate the results. In essence, we have defined map as the transformation of a reducing function. The functions that transform one reducing function into another are called transducers in ClojureScript.

To ilustrate the generality of transducers, let’s use different sources and destinations in our call to reduce:

(reduce (incer str) "" [0 1 2])
;; => "123"

(reduce (only-odds str) "" '(0 1 2))
;; => "1"

The transducer versions of map and filter transform a process that carries inputs from a source to a destination but don’t know anything about where the inputs come from and where they end up. In their implementation they contain the essence of what they accomplish, independent of context.

Now that we know more about transducers we can try to implement our own version of mapcat. We already have a fundamental piece of it: the map transducer. What mapcat does is map a function over an input and flatten the resulting structure one level. Let’s try to implement the catenation part as a transducer:

(defn my-cat
  [rfn]
  (fn [acc input]
    (reduce rfn acc input)))

(reduce (my-cat conj) [] [[0 1 2] [3 4 5]])
;; => [0 1 2 3 4 5]

The my-cat transducer returns a reducing function that catenates its inputs into the accumulator. It does so reducing the input reducible with the rfn reducing function and using the accumulator (acc) as the initial value for such reduction. mapcat is simply the composition of map and cat. The order in which transducers are composed may seem backwards but it’ll become clear in a moment.

(defn my-mapcat
  [f]
  (comp (my-mapt f) my-cat))

(defn dupe
  [x]
  [x x])

(def duper (my-mapcat dupe))

(reduce (duper conj) [] [0 1 2])
;; => [0 0 1 1 2 2]

5.2.6. Transducers in ClojureScript core

Some of the ClojureScript core functions like map, filter and mapcat support an arity 1 version that returns a transducer. Let’s revisit our definition of process-cluster and define it in terms of transducers:

(def process-clusters
  (comp
    (mapcat split-cluster)
    (filter not-rotten)
    (map clean-grape)))

A few things changed since our previous definition process-clusters. First of all, we are using the transducer-returning versions of mapcat, filter and map instead of partially applying them for working on sequences.

Also you may have noticed that the order in which they are composed is reversed, they appear in the order they are executed. Note that all map, filter and mapcat return a transducer. filter transforms the reducing function returned by map, applying the filtering before proceeding; mapcat transforms the reducing function returned by filter, applying the mapping and catenation before proceeding.

One of the powerful properties of transducers is that they are combined using regular function composition. What’s even more elegant is that the composition of various transducers is itself a transducer! This means that our process-cluster is a transducer too, so we have defined a composable and context-independent algorithmic transformation.

Many of the core ClojureScript functions accept a transducer, let’s look at some examples with our newly created process-cluster:

(into [] process-clusters grape-clusters)
;; => [{:rotten? false, :clean? true} {:rotten? false, :clean? true}]

(sequence process-clusters grape-clusters)
;; => ({:rotten? false, :clean? true} {:rotten? false, :clean? true})

(reduce (process-clusters conj) [] grape-clusters)
;; => [{:rotten? false, :clean? true} {:rotten? false, :clean? true}]

Since using reduce with the reducing function returned from a transducer is so common, there is a function for reducing with a transformation called transduce. We can now rewrite the previous call to reduce using transduce:

(transduce process-clusters conj [] grape-clusters)
;; => [{:rotten? false, :clean? true} {:rotten? false, :clean? true}]

5.2.7. Initialisation

In the last example we provided an initial value to the transduce function ([]) but we can omit it and get the same result:

(transduce process-clusters conj grape-clusters)
;; => [{:rotten? false, :clean? true} {:rotten? false, :clean? true}]

What’s going on here? How can transduce know what initial value use as an accumulator when we haven’t specified it? Try calling conj without any arguments and see what happens:

(conj)
;; => []

The conj function has a arity 0 version that returns an empty vector but is not the only reducing function that supports arity 0. Let’s explore some others:

(+)
;; => 0

(*)
;; => 1

(str)
;; => ""

(= identity (comp))
;; => true

The reducing function returned by transducers must support the arity 0 as well, which will typically delegate to the transformed reducing function. There is no sensible implementation of the arity 0 for the transducers we have implemented so far, so we’ll simply call the reducing function without arguments. Here’s how our modified my-mapt could look like:

(defn my-mapt
  [f]
  (fn [rfn]
    (fn
      ([] (rfn))                ;; arity 0 that delegates to the reducing fn
      ([acc input]
        (rfn acc (f input))))))

The call to the arity 0 of the reducing function returned by a transducer will call the arity 0 version of every nested reducing function, eventually calling the outermost reducing function. Let’s see an example with our already defined process-clusters transducer:

((process-clusters conj))
;; => []

The call to the arity 0 flows through the transducer stack, eventually calling (conj).

5.2.8. Stateful transducers

So far we’ve only seen purely functional transducer; they don’t have any implicit state and are very predictable. However, there are many data transformation functions that are inherently stateful, like take. take receives a number n of elements to keep and a collection and returns a collection with at most n elements.

(take 10 (range 100))
;; => (0 1 2 3 4 5 6 7 8 9)

Let’s step back for a bit and learn about the early termination of the reduce function. We can wrap an accumulator in a type called reduced for telling reduce that the reduction process should terminate immediately. Let’s see an example of a reduction that aggregates the inputs in a collection and finishes as soon as there are 10 elements in the accumulator:

(reduce (fn [acc input]
          (if (= (count acc) 10)
            (reduced acc)
            (conj acc input)))
         []
         (range 100))
;; => [0 1 2 3 4 5 6 7 8 9]

Since transducers are modifications of reducing functions they also use reduced for early termination. Note that stateful transducers may need to do some cleanup before the process terminates, so they must support an arity 1 as a "completion" step. Usually, like with arity 0, this arity will simply delegate to the transformed reducing function’s arity 1.

Knowing this we are able to write stateful transducers like take. We’ll be using mutable state internally for tracking the number of inputs we have seen so far, and wrap the accumulator in a reduced as soon as we’ve seen enough elements:

(defn my-take
  [n]
  (fn [rfn]
    (let [remaining (volatile! n)]
      (fn
        ([] (rfn))
        ([acc] (rfn acc))
        ([acc input]
          (let [rem @remaining
                nr (vswap! remaining dec)
                result (if (pos? rem)
                         (rfn acc input)   ;; we still have items to take
                         acc)]             ;; we're done, acc becomes the result
            (if (not (pos? nr))
              (ensure-reduced result)      ;; wrap result in reduced if not already
              result)))))))

This is a simplified version of the take function present in ClojureScript core. There are a few things to note here so let’s break it up in pieces to understand it better.

The first thing to notice is that we are creating a mutable value inside the transducer. Note that we don’t create it until we receive a reducing function to transform. If we created it before returning the transducer we couldn’t use my-take more than once. Since the transducer is handed a reducing function to transform each time it is used, we can use it multiple times and the mutable variable will be created in every use.

(fn [rfn]
  (let [remaining (volatile! n)] ;; make sure to create mutable variables inside the transducer
    (fn
      ;; ...
)))

(def take-five (my-take 5))

(transduce take-five conj (range 100))
;; => [0 1 2 3 4]

(transduce take-five conj (range 100))
;; => [0 1 2 3 4]

Let’s now dig into the reducing function returned from my-take. First of all we deref the volatile to get the number of elements that remain to be taken and decrement it to get the next remaining value. If there are still remaining items to take, we call rfn passing the accumulator and input; if not, we already have the final result.

([acc input]
  (let [rem @remaining
        nr (vswap! remaining dec)
        result (if (pos? rem)
                 (rfn acc input)
                 acc)]
    ;; ...
))

The body of my-take should be obvious by now. We check whether there are still items to be processed using the next remainder (nr) and, if not, wrap the result in a reduced using the ensure-reduced function. ensure-reduced will wrap the value in a reduced if it’s not reduced already or simply return the value if it’s already reduced. In case we are not done yet, we return the accumulated result for further processing.

(if (not (pos? nr))
  (ensure-reduced result)
  result)

We’ve seen an example of a stateful transducer but it didn’t do anything in its completion step. Let’s see an example of a transducer that uses the completion step to flush an accumulated value. We’ll implement a simplified version of partition-all, which given a n number of elements converts the inputs in vectors of size n. For understanding its purpose better let’s see what the arity 2 version gives us when providing a number and a collection:

(partition-all 3 (range 10))
;; => ((0 1 2) (3 4 5) (6 7 8) (9))

The transducer returning function of partition-all will take a number n and return a transducer that groups inputs in vectors of size n. In the completion step it will check if there is an accumulated result and, if so, add it to the result. Here’s a simplified version of ClojureScript core partition-all function, where array-list is a wrapper for a mutable JavaScript array:

(defn my-partition-all
  [n]
  (fn [rfn]
    (let [a (array-list)]
      (fn
        ([] (rfn))
        ([result]
          (let [result (if (.isEmpty a)                  ;; no inputs accumulated, don't have to modify result
                         result
                         (let [v (vec (.toArray a))]
                           (.clear a)                    ;; flush array contents for garbage collection
                           (unreduced (rfn result v))))] ;; pass to `rfn`, removing the reduced wrapper if present
            (rfn result)))
        ([acc input]
          (.add a input)
          (if (== n (.size a))                           ;; got enough results for a chunk
            (let [v (vec (.toArray a))]
              (.clear a)
              (rfn acc v))                               ;; the accumulated chunk becomes input to `rfn`
            acc))))))

(def triples (my-partition-all 3))

(transduce triples conj (range 10))
;; => [[0 1 2] [3 4 5] [6 7 8] [9]]

5.2.9. Eductions

Eductions are a way to combine a collection and one or more transformations that can be reduced and iterated over, and that apply the transformations every time we do so. If we have a collection that we want to process and a transformation over it that we want others to extend, we can hand them a eduction, encapsulating the source collection and our transformation. We can create an eduction with the eduction function:

(def ed (eduction (filter odd?) (take 5) (range 100)))

(reduce + 0 ed)
;; => 25

(transduce (partition-all 2) conj ed)
;; => [[1 3] [5 7] [9]]

5.2.10. More transducers in ClojureScript core

We learned about map, filter, mapcat, take and partition-all, but there are a lot more transducers available in ClojureScript. Here is an incomplete list of some other intersting ones:

  • drop is the dual of take, dropping up to n values before passing inputs to the reducing function

  • distinct only allows inputs to occur once

  • dedupe removes succesive duplicates in input values

We encourage you to explore ClojureScript core to see what other transducers are out there.

5.2.11. Defining our own transducers

There a few things to consider before writing our own transducers so in this section we will learn how to properly implement one. First of all, we’ve learned that the general structure of a transducer is the following:

(fn [xf]
  (fn
    ([]          ;; init
      ...)
    ([r]         ;; completion
      ...)
    ([acc input] ;; step
      ...)))

Usually only the code represented with …​ changes between transducers, these are the invariants that must be preserved in each arity of the resulting function:

  • arity 0 (init): must call the arity 0 of the nested transform xf

  • arity 1 (completion): used to produce a final value and potentially flush state, must call the arity 1 of the nested transform xf exactly once

  • arity 2 (step): the resulting reducing function which will call the arity 2 of the nested transform xf zero, one or more times

5.2.12. Transducible processes

A transducible process is any process that is defined in terms of a succession of steps ingesting input values. The source of input varies from one process to another. Most of our examples dealt with inputs from a collection or a lazy sequence, but it could be an asynchronous stream of values or a core.async channel. The outputs produced by each step are also different for every process; into creates a collection with every output of the transducer, sequence yields a lazy sequence, and asynchronous streams would probably push the outputs to their listeners.

In order to improve our understanding of transducible processes, we’re going to implement an unbounded queue, since adding values to a queue can be thought in terms of a succession of steps ingesting input. First of all we’ll define a protocol and a data type that implements the unbounded queue:

(defprotocol Queue
  (put! [q item] "put an item into the queue")
  (take! [q] "take an item from the queue")
  (shutdown! [q] "stop accepting puts in the queue"))

(deftype UnboundedQueue [^:mutable arr ^:mutable closed]
  Queue
  (put! [_ item]
    (assert (not closed))
    (assert (not (nil? item)))
    (.push arr item)
    item)
  (take! [_]
    (aget (.splice arr 0 1) 0))
  (shutdown! [_]
    (set! closed true)))

We defined the Queue protocol and as you may have noticed the implementation of UnboundedQueue doesn’t know about transducers at all. It has a put! operation as its step function and we’re going to implement the transducible process on top of that interface:

(defn unbounded-queue
  ([]
   (unbounded-queue nil))
  ([xform]
   (let [put! (completing put!)
         xput! (if xform (xform put!) put!)
         q (UnboundedQueue. #js [] false)]
     (reify
       Queue
       (put! [_ item]
         (when-not (.-closed q)
           (let [val (xput! q item)]
             (if (reduced? val)
               (do
                 (xput! @val)  ;; call completion step
                 (shutdown! q) ;; respect reduced
                 @val)
               val))))
       (take! [_]
         (take! q))
       (shutdown! [_]
         (shutdown! q))))))

As you can see, the unbounded-queue constructor uses an UnboundedQueue instance internally, proxying the take! and shutdown! calls and implementing the transducible process logic in the put! function. Let’s deconstruct it to understand what’s going on.

(let [put! (completing put!)
      xput! (if xform (xform put!) put!)
      q (UnboundedQueue. #js [] false)]
  ;; ...
)

First of all, we use completing for adding the arity 0 and arity 1 to the Queue protocol’s put! function. This will make it play nicely with transducers in case we give this reducing function to xform to derive another. After that, if a transducer (xform) was provided, we derive a reducing function applying the transducer to put!. If we’re not handed a transducer we will just use put!. q is the internal instance of UnboundedQueue.

(reify
  Queue
  (put! [_ item]
    (when-not (.-closed q)
      (let [val (xput! q item)]
        (if (reduced? val)
          (do
            (xput! @val)  ;; call completion step
            (shutdown! q) ;; respect reduced
            @val)
          val))))
  ;; ...
)

The exposed put! operation will only be performed if the queue hasn’t been shut down. Notice that the put! implementation of UnboundedQueue uses an assert to verify that we can still put values to it and we don’t want to break that invariant. If the queue isn’t closed we can put values into it, we use the possibly transformed xput! for doing so.

If the put operation gives back a reduced value it’s telling us that we should terminate the transducible process. In this case that means shutting down the queue to not accept more values. If we didn’t get a reduced value we can happily continue accepting puts.

Let’s see how our queue behaves without transducers:

(def q (unbounded-queue))
;; => #<[object Object]>

(put! q 1)
;; => 1
(put! q 2)
;; => 2

(take! q)
;; => 1
(take! q)
;; => 2
(take! q)
;; => nil

Pretty much what we expected, let’s now try with a stateless transducer:

(def incq (unbounded-queue (map inc)))
;; => #<[object Object]>

(put! incq 1)
;; => 2
(put! incq 2)
;; => 3

(take! incq)
;; => 2
(take! incq)
;; => 3
(take! incq)
;; => nil

To check that we’ve implemented the transducible process, let’s use a stateful transducer. We’ll use a transducer that will accept values while they aren’t equal to 4 and will partition inputs in chunks of 2 elements:

(def xq (unbounded-queue (comp
                           (take-while #(not= % 4))
                           (partition-all 2))))

(put! xq 1)
(put! xq 2)
;; => [1 2]
(put! xq 3)
(put! xq 4) ;; shouldn't accept more values from here on
(put! xq 5)
;; => nil

(take! xq)
;; => [1 2]
(take! xq) ;; seems like `partition-all` flushed correctly!
;; => [3]
(take! xq)
;; => nil

The example of the queue was heavily inspired by how core.async channels use transducers in their internal step. We’ll discuss channels and their usage with transducers in a later section.

Transducible processes must respect reduced as a way for signaling early termination. For example, building a collection stops when encountering a reduced and core.async channels with transducers are closed. The reduced value must be unwrapped with deref and passed to the completion step, which must be called exactly once.

Transducible processes shouldn’t expose the reducing function they generate when calling the transducer with their own step function since it may be stateful and unsafe to use from elsewhere.

5.3. Transients

Although ClojureScript’s immutable and persistent data structures are reasonably performant there are situations in which we are transforming large data structures using multiple steps to only share the final result. For example, the core into function takes a collection and eagerly populates it with the contents of a sequence:

(into [] (range 100))
;; => [0 1 2 ... 98 99]

In the above example we are generating a vector of 100 elements conj-ing one at a time. Every intermediate vector that is not the final result won’t be seen by anybody except the into function and the array copying required for persistence is an unnecesary overhead.

For these situations ClojureScript provides a special version of some of its persistent data structures, which are called transients. Maps, vectors and sets have a transient counterpart. Transients are always derived from a persistent data structure using the transient function, which creates a transient version in constant time:

(def tv (transient [1 2 3]))
;; => #<[object Object]>

Transients support the read API of their persistent counterparts:

(def tv (transient [1 2 3]))

(nth tv 0)
;; => 1

(get tv 2)
;; => 3

(def tm (transient {:language "ClojureScript"}))

(:language tm)
;; => "ClojureScript"

(def ts (transient #{:a :b :c}))

(contains? ts :a)
;; => true

(:a ts)
;; => :a

Since transients don’t have persistent and immutable semantics for updates they can’t be transformed using the already familiar conj or assoc functions. Instead, the transforming functions that work on transients end with a bang. Let’s look at an example using conj! on a transient:

(def tv (transient [1 2 3]))

(conj! tv 4)
;; => #<[object Object]>

(nth tv 3)
;; => 4

As you can see, the transient version of the vector is neither immutable nor persistent. Instead, the vector is mutated in place. Although we could transform tv repeatedly using conj! on it we shouldn’t abandon the idioms used with the persistent data structures: when transforming a transient, use the returned version of it for further modifications like in the following example:

(-> [1 2 3]
  transient
  (conj! 4)
  (conj! 5))
;; => #<[object Object]>

We can convert a transient back to a persistent and immutable data structure by calling persistent! on it. This operation, like deriving a transient from a persistent data structure, is done in constant time.

(-> [1 2 3]
  transient
  (conj! 4)
  (conj! 5)
  persistent!)
;; => [1 2 3 4 5]

A peculiarity of transforming transients into persistent structures is that the transient version is invalidated after being converted to a persistent data structure and we can’t do further transformations to it. This happens because the derived persistent data structure uses the transient’s internal nodes and mutating them would break the immutability and persistent guarantees:

(def tm (transient {}))
;; => #<[object Object]>

(assoc! tm :foo :bar)
;; => #<[object Object]>

(persistent! tm)
;; => {:foo :bar}

(assoc! tm :baz :frob)
;; Error: assoc! after persistent!

Going back to our initial example with into, here’s a very simplified implementation of it that uses a transient for performance, returning a persistent data structure and thus exposing a purely functional interface although it uses mutation internally:

(defn my-into
  [to from]
  (persistent! (reduce conj! (transient to) from)))

(my-into [] (range 100))
;; => [0 1 2 ... 98 99]

5.4. Metadata

ClojureScript symbols, vars and persistent collections support attaching metadata to them. Metadata is a map with information about the entity it’s attached to. The ClojureScript compiler uses metadata for several purposes such as type hints, and the metadata system can be used by tooling, library and application developers too.

There may not be many cases in day-to-day ClojureScript programming where you need metadata, but it is a nice language feature to have and know about; it may come in handy at some point. It makes things like runtime code introspection and documentation generation very easy. You’ll see why throughout this section.

5.4.1. Vars

Let’s define a var and see what metadata is attached to it by default. Note that this code is executed in a REPL, and thus the metadata of a var defined in a source file may vary. We’ll use the meta function to retrieve the metadata of the given value:

(def answer-to-everything 42)
;; => 42

#'answer-to-everything
;; => #'cljs.user/answer-to-everyhing

(meta #'answer-to-everything)
;; => {:ns cljs.user,
;;     :name answer-to-everything,
;;     :file "NO_SOURCE_FILE",
;;     :source "answer-to-everything",
;;     :column 6,
;;     :end-column 26,
;;     :line 1,
;;     :end-line 1,
;;     :arglists (),
;;     :doc nil,
;;     :test nil}

Few things to note here. First of all, #'answer-to-everything gives us a reference to the Var that holds the value of the answer-to-everything symbol. We see that it includes information about the namespace (:ns) in which it was defined, its name, file (although, since it was defined at a REPL doesn’t have a source file), source, position in the file where it was defined, argument list (which only makes sense for functions), documentation string and test function.

Let’s take a look at a function var’s metadata:

(defn add
  "A function that adds two numbers."
  [x y]
  (+ x y))

(meta #'add)
;; => {:ns cljs.user,
;;     :name add,
;;     :file "NO_SOURCE_FILE",
;;     :source "add",
;;     :column 7,
;;     :end-column 10,
;;     :line 1,
;;     :end-line 1,
;;     :arglists (quote ([x y])),
;;     :doc "A function that adds two numbers.",
;;     :test nil}

We see that the argument lists are stored in the :arglists field of the var’s metadata and its documentation in the :doc field. We’ll now define a test function to learn about what :test is used for:

(require '[cljs.test :as t])

(t/deftest i-pass
  (t/is true))

(meta #'i-pass)
;; => {:ns cljs.user,
;;     :name i-pass,
;;     :file "NO_SOURCE_FILE",
;;     :source "i-pass",
;;     :column 12,
;;     :end-column 18,
;;     :line 1,
;;     :end-line 1,
;;     :arglists (),
;;     :doc "A function that adds two numbers.",
;;     :test #<function (){ ... }>}

The :test attribute (truncated for brevity) in the i-pass var’s metadata is a test function. This is used by the cljs.test library for discovering and running tests in the namespaces you tell it to.

5.4.2. Values

We learned that vars can have metadata and what kind of metadata is added to them for consumption by the compiler and the cljs.test testing library. Persistent collections can have metadata too, although they don’t have any by default. We can use the with-meta function to derive an object with the same value and type with the given metadata attached. Let’s see how:

(def map-without-metadata {:language "ClojureScript"})
;; => {:language "ClojureScript"}

(meta map-without-metadata)
;; => nil

(def map-with-metadata (with-meta map-without-metadata
                                  {:answer-to-everything 42}))
;; => {:language "ClojureScript"}

(meta map-with-metadata)
;; => {:answer-to-everything 42}

(= map-with-metadata
   map-without-metadata)
;; => true

(identical? map-with-metadata
            map-without-metadata)
;; => false

It shouldn’t come as a surprise that metadata doesn’t affect equality between two data structures since equality in ClojureScript is based on value. Another interesting thing is that with-meta creates another object of the same type and value as the given one and attaches the given metadata to it.

Another open question is what happens with metadata when deriving new values from a persistent data structure. Let’s find out:

(def derived-map (assoc map-with-metadata :language "Clojure"))
;; => {:language "Clojure"}

(meta derived-map)
;; => {:answer-to-everything 42}

As you can see in the example above, metadata is preserved in derived versions of persistent data structures. There are some subtleties, though. As long as the functions that derive new data structures return collections with the same type, metadata will be preserved; this is not true if the types change due to the transformation. To ilustrate this point, let’s see what happens when we derive a seq or a subvector from a vector:

(def v (with-meta [0 1 2 3] {:foo :bar}))
;; => [0 1 2 3]

(def sv (subvec v 0 2))
;; => [0 1]

(meta sv)
;; => nil

(meta (seq v))
;; => nil

5.4.3. Syntax for metadata

The ClojureScript reader has syntactic support for metadata annotations, which can be written in different ways. We can prepend var definitions or collections with a caret (^) followed by a map for annotating it with the given metadata map:

(def ^{:doc "The answer to Life, Universe and Everything."} answer-to-everything 42)
;; => 42

(meta #'answer-to-everything)
;; => {:ns cljs.user,
;;     :name answer-to-everything,
;;     :file "NO_SOURCE_FILE",
;;     :source "answer-to-everything",
;;     :column 6,
;;     :end-column 26,
;;     :line 1,
;;     :end-line 1,
;;     :arglists (),
;;     :doc "The answer to Life, Universe and Everything.",
;;     :test nil}

(def map-with-metadata ^{:answer-to-everything 42} {:language "ClojureScript"})
;; => {:language "ClojureScript"}

(meta map-with-metadata)
;; => {:answer-to-everything 42}

Notice how the metadata given in the answer-to-everything var definition is merged with the var metadata.

A very common use of metadata is to set certain keys to a true value. For example we may want to add to a var’s metadata that the variable is dynamic or a constant. For such cases, we have a shorthand notation that uses a caret followed by a keyword. Here are some examples:

(def ^:dynamic *foo* 42)
;; => 42

(:dynamic (meta #'*foo*))
;; => true

(def ^:foo ^:bar answer 42)
;; => 42

(select-keys (meta #'answer) [:foo :bar])
;; => {:foo true, :bar true}

There is another shorthand notation for attaching metadata. If we use a caret followed by a symbol it will be added to the metadata map under the :tag key. Using tags such as ^boolean gives the ClojureScript compiler hints about the type of expressions or function return types.

(defn ^boolean will-it-blend? [_] true)
;; => #<function ... >

(:tag (meta #'will-it-blend?))
;; => boolean

(not ^boolean (js/isNaN js/NaN))
;; => false

5.4.4. Functions for working with metadata

We’ve learned about meta and with-meta so far but ClojureScript offers a few functions for transforming metadata. There is vary-meta which is similar to with-meta in that it derives a new object with the same type and value as the original but it doesn’t take the metadata to attach directly. Instead, it takes a function to apply to the metadata of the given object to transform it for deriving new metadata. This is how it works:

(def map-with-metadata ^{:foo 40} {:language "ClojureScript"})
;; => {:language "ClojureScript"}

(meta map-with-metadata)
;; => {:foo 40}

(def derived-map (vary-meta map-with-metadata update :foo + 2))
;; => {:language "ClojureScript"}

(meta derived-map)
;; => {:foo 42}

If instead we want to change the metadata of an existing var or value we can use alter-meta! for changing it by applying a function or reset-meta! for replacing it with another map:

(def map-with-metadata ^{:foo 40} {:language "ClojureScript"})
;; => {:language "ClojureScript"}

(meta map-with-metadata)
;; => {:foo 40}

(alter-meta! map-with-metadata update :foo + 2)
;; => {:foo 42}

(meta map-with-metadata)
;; => {:foo 42}

(reset-meta! map-with-metadata {:foo 40})
;; => {:foo 40}

(meta map-with-metadata)
;; => {:foo 40}

5.5. Core protocols

One of the greatest qualities of the core ClojureScript functions is that they are implemented around protocols. This makes them open to work on any type that we extend with such protocols, be it defined by us or a third party.

5.5.1. Functions

As we’ve learned in previous chapters not only functions can be invoked. Vectors are functions of their indexes, maps are functions of their keys and sets are functions of their values.

We can extend types to be callable as functions implementing the IFn protocol. A collection that doesn’t support calling it as a function is the queue, let’s implement IFn for the PersistentQueue type so we’re able to call queues as functions of their indexes:

(extend-type PersistentQueue
  IFn
  (-invoke
    ([this idx]
      (nth this idx))))

(def q #queue[:a :b :c])
;; => #queue [:a :b :c]

(q 0)
;; => :a

(q 1)
;; => :b

(q 2)
;; => :c

5.5.2. Printing

For learning about some of the core protocols we’ll define a Pair type, which will hold a pair of values.

(deftype Pair [fst snd])

If we want to customize how types are printed we can implement the IPrintWithWriter protocol. It defines a function called -pr-writer that receives the value to print, a writer object and options; this function uses the writer object’s -write function to write the desired Pair string representation:

(extend-type Pair
  IPrintWithWriter
  (-pr-writer [p writer _]
    (-write writer (str "#<Pair " (.-fst p) "," (.-snd p) ">"))))

5.5.3. Sequences

In a previous section we learned about sequences, one of ClojureScript’s main abstractions. Remember the first and rest functions for working with sequences? They are defined in the ISeq protocol, so we can extend types for responding to such functions:

(extend-type Pair
  ISeq
  (-first [p]
    (.-fst p))

  (-rest [p]
    (list (.-snd p))))

(def p (Pair. 1 2))
;; => #<Pair 1,2>

(first p)
;; => 1

(rest p)
;; => (2)

Another handy function for working with sequences is next. Although next works as long as the given argument is a sequence, we can implement it explicitly with the INext protocol:

(def p (Pair. 1 2))

(next p)
;; => (2)

(extend-type Pair
  INext
  (-next [p]
    (println "Our next")
    (list (.-snd p))))

(next p)
;; Our next
;; => (2)

Finally, we can make our own types seqable implementing the ISeqable protocol. This means we can pass them to seq for getting a sequence back.

ISeqable

(def p (Pair. 1 2))

(extend-type Pair
  ISeqable
  (-seq [p]
    (list (.-fst p) (.-snd p))))

(seq p)
;; => (1 2)

Now our Pair type works with the plethora of ClojureScript functions for working with sequences:

(def p (Pair. 1 2))
;; => #<Pair 1,2>

(map inc p)
;; => (2 3)

(filter odd? p)
;; => (1)

(reduce + p)
;; => 3

5.5.4. Collections

Collection functions are also defined in terms of protocols. For this section examples we will make the native JavaScript string act like a collection.

The most important function for working with collection is conj, defined in the ICollection protocol. Strings are the only type which makes sense to conj to a string, so the conj operation for strings will be simply a concatenation:

(extend-type string
  ICollection
  (-conj [this o]
    (str this o)))

(conj "foo" "bar")
;; => "foobar"

(conj "foo" "bar" "baz")
;; => "foobarbaz"

Another handy function for working with collections is empty, which is part of the IEmptyableCollection protocol. Let’s implement it for the string type:

(extend-type string
  IEmptyableCollection
  (-empty [_]
    ""))

(empty "foo")
;; => ""

We used the string special symbol for extending the native JavaScript string. If you want to learn more about it check the section about extending JavaScript types.

Collection traits

There are some qualities that not all collections have, such as being countable in constant time or being reversible. These traits are splitted into different protocols since not all of them make sense for every collection. For illustrating these protocols we’ll use the Pair type we defined earlier.

For collections that can be counted in constant time using the count function we can implement the ICounted protocol. It should be easy to implement it for the Pair type:

(extend-type Pair
  ICounted
  (-count [_]
    2))

(def p (Pair. 1 2))

(count p)
;; => 2

Some collection types such as vectors and lists can be indexed by a number using the nth function. If our types are indexed we can implement the IIndexed protocol:

(extend-type Pair
  IIndexed
  (-nth
    ([p idx]
      (case idx
        0 (.-fst p)
        1 (.-snd p)
        (throw (js/Error. "Index out of bounds"))))
    ([p idx default]
      (case idx
        0 (.-fst p)
        1 (.-snd p)
        default))))

(nth p 0)
;; => 1

(nth p 1)
;; => 2

(nth p 2)
;; Error: Index out of bounds

(nth p 2 :default)
;; => :default

5.5.5. Associative

There are many data structures that are associative: they map keys to values. We’ve encountered a few of them already and we know many functions for working with them like get, assoc or dissoc. Let’s explore the protocols that these functions build upon.

First of all, we need a way to look up keys on an associative data structure. The ILookup protocol defines a function for doing so, let’s add the ability to look up keys in our Pair type since it is an associative data structure that maps the indices 0 and 1 to values.

(extend-type Pair
  ILookup
  (-lookup
    ([p k]
      (-lookup p k nil))
    ([p k default]
      (case k
        0 (.-fst p)
        1 (.-snd p)
        default))))

(get p 0)
;; => 1

(get p 1)
;; => 2

(get p :foo)
;; => nil

(get p 2 :default)
;; => :default

For using assoc on a data structure it must implement the IAssociative protocol. For our Pair type only 0 and 1 will be allowed as keys for associating values. IAssociative also has a function for asking whether a key is present or not.

(extend-type Pair
  IAssociative
  (-contains-key? [_ k]
    (contains? #{0 1} k))

  (-assoc [p k v]
    (case k
      0 (Pair. v (.-snd p))
      1 (Pair. (.-fst p) v)
      (throw (js/Error. "Can only assoc to 0 and 1 keys")))))

(def p (Pair. 1 2))
;; => #<Pair 1,2>

(assoc p 0 2)
;; => #<Pair 2,2>

(assoc p 1 1)
;; => #<Pair 1,1>

(assoc p 0 0 1 1)
;; => #<Pair 0,1>

(assoc p 2 3)
;; Error: Can only assoc to 0 and 1 keys

The complementary function for assoc is dissoc and it’s part of the IMap protocol. It doesn’t make much sense for our Pair type but we’ll implement it nonetheless. Dissociating 0 or 1 will mean putting a nil in such position and invalid keys will be ignored.

(extend-type Pair
  IMap
  (-dissoc [p k]
    (case k
      0 (Pair. nil (.-snd p))
      1 (Pair. (.-fst p) nil)
      p)))

(def p (Pair. 1 2))
;; => #<Pair 1,2>

(dissoc p 0)
;; => #<Pair ,2>

(dissoc p 1)
;; => #<Pair 1,>

(dissoc p 2)
;; => #<Pair 1,2>

(dissoc p 0 1)
;; => #<Pair ,>

Associative data structures are made of key and value pairs we can call entries. The key and val functions allow us to query the key and value of such entries and they are built upon the IMapEntry protocol. Let’s see a few examples of key and val and how map entries can be used to build up maps:

(key [:foo :bar])
;; => :foo

(val [:foo :bar])
;; => :bar

(into {} [[:foo :bar] [:baz :xyz]])
;; => {:foo :bar, :baz :xyz}

Pairs can be map entries too, we treat their first element as the key and the second as the value:

(extend-type Pair
  IMapEntry
  (-key [p]
    (.-fst p))

  (-val [p]
    (.-snd p)))

(def p (Pair. 1 2))
;; => #<Pair 1,2>

(key p)
;; => 1

(val p)
;; => 2

(into {} [p])
;; => {1 2}

5.5.6. Comparison

For checking whether two or more values are equivalent with = we must implement the IEquiv protocol. Let’s do it for our Pair type:

(def p  (Pair. 1 2))
(def p' (Pair. 1 2))
(def p'' (Pair. 1 2))

(= p p')
;; => false

(= p p' p'')
;; => false

(extend-type Pair
  IEquiv
  (-equiv [p other]
    (and (instance? Pair other)
         (= (.-fst p) (.-fst other))
         (= (.-snd p) (.-snd other)))))

(= p p')
;; => true

(= p p' p'')
;; => true

We can also make types comparable. The function compare takes two values and returns a negative number if the first is less than the second, 0 if both are equal and 1 if the first is greater than the second. For making our types comparable we must implement the IComparable protocol.

For pairs, comparison will mean checking if the two first values are equal. If they are, the result will be the comparison of the second values. If not, we will return the result of the first comparison:

(extend-type Pair
  IComparable
  (-compare [p other]
    (let [fc (compare (.-fst p) (.-fst other))]
      (if (zero? fc)
        (compare (.-snd p) (.-snd other))
        fc))))

(compare (Pair. 0 1) (Pair. 0 1))
;; => 0

(compare (Pair. 0 1) (Pair. 0 2))
;; => -1

(compare (Pair. 1 1) (Pair. 0 2))
;; => 1

(sort [(Pair. 1 1) (Pair. 0 2) (Pair. 0 1)])
;; => (#<Pair 0,1> #<Pair 0,2> #<Pair 1,1>)

5.5.7. Metadata

The meta and with-meta functions are also based upon two protocols: IMeta and IWithMeta respectively. We can make our own types capable of carrying metadata adding an extra field for holding the metadata and implementing both protocols.

Let’s implement a version of Pair that can have metadata:

(deftype Pair [fst snd meta]
  IMeta
  (-meta [p] meta)

  IWithMeta
  (-with-meta [p new-meta]
    (Pair. fst snd new-meta)))


(def p (Pair. 1 2 {:foo :bar}))
;; => #<Pair 1,2>

(meta p)
;; => {:foo :bar}

(def p' (with-meta p {:bar :baz}))
;; => #<Pair 1,2>

(meta p')
;; => {:bar :baz}

5.5.8. Interoperability with JavaScript

Since ClojureScript is hosted in a JavaScript VM we often need to convert ClojureScript data structures to JavaScript ones and viceversa. We also may want to make native JS types participate in an abstraction represented by a protocol.

Extending JavaScript types

When extending JavaScript objects instead of using JS globals like js/String, js/Date and such, special symbols are used. This is done for avoiding mutating global JS objects.

The symbols for extending JS types are: object, array, number, string, function, boolean and nil is used for the null object. The dispatch of the protocol to native objects uses Google Closure’s goog.typeOf function. There’s a special default symbol that can be used for making a default implementation of a protocol for every type.

For illustrating the extension of JS types we are going to define a MaybeMutable protocol that’ll have a mutable? predicate as its only function. Since in JavaScript mutability is the default we’ll extend the default JS type returning true from mutable?:

(defprotocol MaybeMutable
  (mutable? [this] "Returns true if the value is mutable."))

(extend-type default
  MaybeMutable
  (mutable? [_] true))

;; object
(mutable? #js {})
;; => true

;; array
(mutable? #js [])
;; => true

;; string
(mutable? "")
;; => true

;; function
(mutable? (fn [x] x))
;; => true

Since fortunately not all JS object’s values are mutable we can refine the implementation of MaybeMutable for returning false for strings and functions.

(extend-protocol MaybeMutable
  string
  (mutable? [_] false)

  function
  (mutable? [_] false))


;; object
(mutable? #js {})
;; => true

;; array
(mutable? #js [])
;; => true

;; string
(mutable? "")
;; => false

;; function
(mutable? (fn [x] x))
;; => false

There is no special symbol for JavaScript dates so we have to extend js/Date directly. The same applies to the rest of the types found in the global js namespace.

Converting data

For converting values from ClojureScript types to JavaScript ones and viceversa we use the clj→js and js→clj functions, which are based in the IEncodeJS and IEncodeClojure protocols respectively.

For the examples we’ll use the Set type introduced in ES6. Note that is not available in every JS runtime.

From ClojureScript to JS

First of all we’ll extend ClojureScript’s set type for being able to convert it to JS. By default sets are converted to JavaScript arrays:

(clj->js #{1 2 3})
;; => #js [1 3 2]

Let’s fix it, clj→js is supposed to convert values recursively so we’ll make sure to convert all the set contents to JS and creating the set with the converted values:

(extend-type PersistentHashSet
  IEncodeJS
  (-clj->js [s]
    (js/Set. (into-array (map clj->js s)))))

(def s (clj->js #{1 2 3}))
(es6-iterator-seq (.values s))
;; => (1 3 2)

(instance? js/Set s)
;; => true

(.has s 1)
;; => true
(.has s 2)
;; => true
(.has s 3)
;; => true
(.has s 4)
;; => false

The es6-iterator-seq is an experimental function in ClojureScript core for obtaining a seq from an ES6 iterable.

From JS to ClojureScript

Now it’s time to extend the JS set to convert to ClojureScript. As with clj→js, js→clj recursively converts the value of the data structure:

(extend-type js/Set
  IEncodeClojure
  (-js->clj [s options]
    (into #{} (map js->clj (es6-iterator-seq (.values s))))))

(= #{1 2 3}
   (js->clj (clj->js #{1 2 3})))
;; => true

(= #{[1 2 3] [4 5] [6]}
   (js->clj (clj->js #{[1 2 3] [4 5] [6]})))
;; => true

Note that there is no one-to-one mapping between ClojureScript and JavaScript values. For example, ClojureScript keywords are converted to JavaScript strings when passed to clj→js.

5.5.9. Reductions

The reduce function is based on the IReduce protocol, which enables us to make our own or third-party types reducible. Apart from using them with reduce they will automatically work with transduce too, which will allow us to make a reduction with a transducer.

The JS array is already reducible in ClojureScript:

(reduce + #js [1 2 3])
;; => 6

(transduce (map inc) conj [] [1 2 3])
;; => [2 3 4]

However, the new ES6 Set type isn’t so let’s implement the IReduce protocol. We’ll get an iterator using the Set’s values method and convert it to a seq with the es6-iterator-seq function; after that we’ll delegate to the original reduce function to reduce the obtained sequence.

(extend-type js/Set
  IReduce
  (-reduce
   ([s f]
     (let [it (.values s)]
       (reduce f (es6-iterator-seq it))))
   ([s f init]
     (let [it (.values s)]
       (reduce f init (es6-iterator-seq it))))))

(reduce + (js/Set. #js [1 2 3]))
;; => 6

(transduce (map inc) conj [] (js/Set. #js [1 2 3]))
;; => [2 3 4]

Associative data structures can be reduced with the reduce-kv function, which is based in the IKVReduce protocol. The main difference between reduce and reduce-kv is that the latter uses a three-argument function as a reducer, receiving the accumulator, key and value for each item.

Let’s look at an example, we will reduce a map to a vector of pairs. Note that, since vectors associate indexes to values, they can also be reduced with reduce-kv.

(reduce-kv (fn [acc k v]
             (conj acc [k v]))
           []
           {:foo :bar
            :baz :xyz})
;; => [[:foo :bar] [:baz :xyz]]

We’ll extend the new ES6 map type to support reduce-kv, we’ll do this by getting a sequence of key-value pairs and calling the reducing function with the accumulator, key and value as positional arguments:

(extend-type js/Map
  IKVReduce
  (-kv-reduce [m f init]
   (let [it (.entries m)]
     (reduce (fn [acc [k v]]
               (f acc k v))
             init
             (es6-iterator-seq it)))))

(def m (js/Map.))
(.set m "foo" "bar")
(.set m "baz" "xyz")

(reduce-kv (fn [acc k v]
             (conj acc [k v]))
           []
           m)
;; => [["foo" "bar"] ["baz" "xyz"]]

In both examples we ended up delegating to the reduce function, which is aware of reduced values and terminates when encountering one. Take into account that if you don’t implement these protocols in terms of reduce you will have to check for reduced values for early termination.

5.5.10. Delayed computation

There are some types that have the notion of asynchronous computation, the value they represent may not be realized yet. We can ask whether a value is realized using the realized? predicate.

Let’s ilustrate it with the Delay type, which takes a computation and executes it when the result is needed. When we dereference a delay the computation is run and the delay is realized:

(defn computation []
  (println "running!")
  42)

(def d (delay (computation)))

(realized? d)
;; => false

(deref d)
;; running!
;; => 42

(realized? d)
;; => true

@d
;; => 42

Both realized? and deref sit atop two protocols: IPending and IDeref.

5.5.11. State

The ClojureScript state constructs such as the Atom and the Volatile have different characteristics and semantics, and the operations on them like add-watch, reset! or swap! are backed by protocols.

Atom

For ilustrating such protocols we will implement our own simplified version of an Atom. It won’t support validators nor metadata, but we will be able to:

  • deref the atom for getting its current value

  • reset! the value contained in the atom

  • swap! the atom with a function for transforming its state

deref is based on the IDeref protocol. reset! is based on the IReset protocol and swap! on ISwap. We’ll start by defining a data type and a constructor for our atom implementation:

(deftype MyAtom [^:mutable state ^:mutable watches]
  IPrintWithWriter
  (-pr-writer [p writer _]
    (-write writer (str "#<MyAtom " (pr-str state) ">"))))

(defn my-atom
  ([]
    (my-atom nil))
  ([init]
    (MyAtom. init {})))

(my-atom)
;; => #<MyAtom nil>

(my-atom 42)
;; => #<MyAtom 42>

Note that we’ve marked both the current state of the atom (state) and the map of watchers (watches) with the {:mutable true} metadata. We’ll be modifying them and we’re making this explicit with the annotations.

Our MyAtom type is not very useful yet, we’ll start by implementing the IDeref protocol so we can dereference its current value:

(extend-type MyAtom
  IDeref
  (-deref [a]
    (.-state a)))

(def a (my-atom 42))

@a
;; => 42

Now that we can dereference it we’ll implement the IWatchable protocol, which will let us add and remove watches to our custom atom. We’ll store the watches in the watches map of MyAtom, associating keys to callbacks.

(extend-type MyAtom
  IWatchable
  (-add-watch [a key f]
    (let [ws (.-watches a)]
      (set! (.-watches a) (assoc ws key f))))

  (-remove-watch [a key]
    (let [ws (.-watches a)]
      (set! (.-watches a) (dissoc ws key))))

  (-notify-watches [a oldval newval]
    (doseq [[key f] (.-watches a)]
      (f key a oldval newval))))

We can now add watches to our atom but is not very useful since we still can’t change it. For incorporating change we have to implement the IReset protocol and make sure we notify the watches every time we reset the atom’s value.

(extend-type MyAtom
  IReset
  (-reset! [a newval]
    (let [oldval (.-state a)]
      (set! (.-state a) newval)
      (-notify-watches a oldval newval)
      newval)))

Now let’s check that we got it right. We’ll add a watch, change the atom’s value making sure the watch gets called and then remove it:

(def a (my-atom 41))
;; => #<MyAtom 41>

(add-watch a :log (fn [key a oldval newval]
                    (println {:key key
                              :old oldval
                              :new newval})))
;; => #<MyAtom 41>

(reset! a 42)
;; {:key :log, :old 41, :new 42}
;; => 42

(remove-watch a :log)
;; => #<MyAtom 42>

(reset! a 43)
;; => 43

Our atom is still missing the swapping functionality so we’ll add that now, let’s implement the ISwap protocol. There are four arities for the -swap! method of the protocol since the function passed to swap! may take one, two, three or more arguments:

(extend-type MyAtom
  ISwap
  (-swap!
   ([a f]
    (let [oldval (.-state a)
          newval (f oldval)]
      (reset! a newval)))

   ([a f x]
     (let [oldval (.-state a)
           newval (f oldval x)]
       (reset! a newval)))

   ([a f x y]
     (let [oldval (.-state a)
           newval (f oldval x y)]
       (reset! a newval)))

   ([a f x y more]
     (let [oldval (.-state a)
           newval (apply f oldval x y more)]
       (reset! a newval)))))

We now have a custom implementation of the atom abstraction, let’s test it in the REPL and see if it behaves like we expect:

(def a (my-atom 0))
;; => #<MyAtom 0>

(add-watch a :log (fn [key a oldval newval]
                    (println {:key key
                              :old oldval
                              :new newval})))
;; => #<MyAtom 0>

(swap! a inc)
;; {:key :log, :old 0, :new 1}
;; => 1

(swap! a + 2)
;; {:key :log, :old 1, :new 3}
;; => 3

(swap! a - 2)
;; {:key :log, :old 3, :new 1}
;; => 1

(swap! a + 2 3)
;; {:key :log, :old 1, :new 6}
;; => 6


(swap! a + 4 5 6)
;; {:key :log, :old 6, :new 21}
;; => 21

(swap! a * 2)
;; {:key :log, :old 21, :new 42}
;; => 42

(remove-watch a :log)
;; => #<MyAtom 42>

We did it! We implemented a version of ClojureScript Atom without support for metadata nor validators, extending it to support such features is left as an exercise for the reader. Note that you’ll need to modify the MyAtom type for being able to store metadata and a validator.

Volatile

Volatiles are simpler than atoms in that they don’t support watching for changes. All changes override the previous value much like the mutable variables present in almost every programming language. Volatiles are based on the IVolatile protocol that only defines a method for vreset!, since vswap! is implemented as a macro.

Let’s start by creating our own volatile type and constructor:

(deftype MyVolatile [^:mutable state]
  IPrintWithWriter
  (-pr-writer [p writer _]
    (-write writer (str "#<MyVolatile " (pr-str state) ">"))))

(defn my-volatile
  ([]
    (my-volatile nil))
  ([v]
    (MyVolatile. v)))

(my-volatile)
;; => #<MyVolatile nil>

(my-volatile 42)
;; => #<MyVolatile 42>

Our MyVolatile still needs to support dereferencing and reseting it, let’s implement IDeref and IVolatile, which will enable use to use deref, vreset! and vswap! in our custom volatile:

(extend-type MyVolatile
  IDeref
  (-deref [v]
    (.-state v))

  IVolatile
  (-vreset! [v newval]
    (set! (.-state v) newval)
    newval))

(def v (my-volatile 0))
;; => #<MyVolatile 42>

(vreset! v 1)
;; => 1

@v
;; => 1

(vswap! v + 2 3)
;; => 6

@v
;; => 6

5.5.12. Mutation

In the section about transients we learned about the mutable counterparts of the immutable and persistent data structures that ClojureScript provides. These data structures are mutable, and the operations on them end with a bang (!) to make that explicit. As you may have guessed every operation on transients is based on protocols.

From persistent to transient and viceversa

We’ve learned that we can transform a persistent data structure with the transient function, which is based on the IEditableCollection protocol; for transforming a transient data structure to a persistent one we use persistent!, based on ITransientCollection.

Implementing immutable and persistent data structures and their transient counterparts is out of the scope of this book but we recommend taking a look at ClojureScript’s data structure implementation if you are curious.

Transient vectors and sets

We’ve learned about most of the protocols for transient data structures but we’re missing two: ITransientVector for using assoc! on transient vectors and ITransientSet for using disj! on transient sets.

For illustrating the ITransientVector protocol we’ll extend the JavaScript array type for making it an associative transient data structure:

(extend-type array
  ITransientAssociative
  (-assoc! [arr key val]
    (if (number? key)
      (-assoc-n! arr key val)
      (throw (js/Error. "Array's key for assoc! must be a number."))))

  ITransientVector
  (-assoc-n! [arr n val]
    (.splice arr n 1 val)
    arr))

(def a #js [1 2 3])
;; => #js [1 2 3]

(assoc! a 0 42)
;; => #js [42 2 3]

(assoc! a 1 43)
;; => #js [42 43 3]

(assoc! a 2 44)
;; => #js [42 43 44]

For illustrating the ITransientSet protocol we’ll extend the ES6 Set type for making it a transient set, supporting the conj!, disj! and persistent! operations. Note that we’ve extended the Set type previously for being able to convert it to ClojureScript and we’ll take advantage of that fact.

(extend-type js/Set
  ITransientCollection
  (-conj! [s v]
    (.add s v)
    s)

  (-persistent! [s]
   (js->clj s))

  ITransientSet
  (-disjoin! [s v]
    (.delete s v)
    s))

(def s (js/Set.))

(conj! s 1)
(conj! s 1)
(conj! s 2)
(conj! s 2)

(persistent! s)
;; => #{1 2}

(disj! s 1)

(persistent! s)
;; => #{2}

5.6. CSP (with core.async)

CSP stands for Communicating Sequential Processes, which is a formalism for describing concurrent systems pioneered by C. A. R. Hoare in 1978. It is a concurrency model based on message passing and synchronization through channels. An in-depth look at the theoretical model behind CSP is beyond the scope of this book; instead we’ll focus on presenting the concurrency primitives that core.async offers.

core.async is not part of ClojureScript core but it’s implemented as a library. Even though it is not part of the core language it’s widely used. Many libraries build on top of the core.async primitives, so we think it is worth covering in the book. It’s also a good example of the syntactic abstractions that can be achieved by transforming code with ClojureScript macros, so we’ll jump right in. You’ll need to have core.async installed to run the examples presented in this section.

5.6.1. Channels

Channels are like conveyor belts, we can put and take a single value at a time from them. They can have multiple readers and writers, and they are the fundamental message-passing mechanism of core.async. In order to see how it works, we’ll create a channel to perform some operations on it.

(require '[cljs.core.async :refer [chan put! take!]])

(enable-console-print!)

(def ch (chan))

(take! ch #(println "Got a value:" %))
;; => nil

;; there is a now a pending take operation, let's put something on the channel

(put! ch 42)
;; Got a value: 42
;; => 42

In the above example we created a channel ch using the chan constructor. After that we performed a take operation on the channel, providing a callback that will be invoked when the take operation succeeds. After using put! to put a value on the channel the take operation completed and the "Got a value: 42" string was printed. Note that put! returned the value that was just put to the channel.

The put! function accepts a callback like take! does but we didn’t provide any in the last example. For puts the callback will be called whenever the value we provided has been taken. Puts and takes can happen in any order, let’s do a few puts followed by takes to illustrate the point:

(require '[cljs.core.async :refer [chan put! take!]])

(def ch (chan))

(put! ch 42 #(println "Just put 42"))
;; => true
(put! ch 43 #(println "Just put 43"))
;; => true

(take! ch #(println "Got" %))
;; Got 42
;; Just put 42
;; => nil

(take! ch #(println "Got" %))
;; Got 43
;; Just put 43
;; => nil

You may be asking yourself why the put! operations return true. It signals that the put operation could be performed, even though the value hasn’t yet been taken. Channels can be closed, which will cause the put operations to not succeed:

(require '[cljs.core.async :refer [chan put! close!]])

(def ch (chan))

(close! ch)
;; => nil

(put! ch 42)
;; => false

The above example was the simplest possible situation but what happens with pending operations when a channel is closed? Let’s do a few takes and close the channel and see what happens:

(require '[cljs.core.async :refer [chan put! take! close!]])

(def ch (chan))

(take! ch #(println "Got value:" %))
;; => nil
(take! ch #(println "Got value:" %))
;; => nil

(close! ch)
;; Got value: nil
;; Got value: nil
;; => nil

We see that if the channel is closed all the take! operations receive a nil value. nil in channels is a sentinel value that signals to takers that the channel has been closed. Because of that, putting a nil value on a channel is not allowed:

(require '[cljs.core.async :refer [chan put!]])

(def ch (chan))

(put! ch nil)
;; Error: Assert failed: Can't put nil in on a channel
Buffers

We’ve seen that pending take and put operations are enqueued in a channel but, what happens when there are many pending take or put operations? Let’s find out by hammering a channel with many puts and takes:

(require '[cljs.core.async :refer [chan put! take!]])

(def ch (chan))

(dotimes [n 1025]
  (put! ch n))
;; Error: Assert failed: No more than 1024 pending puts are allowed on a single channel.

(def ch (chan))

(dotimes [n 1025]
  (take! ch #(println "Got" %)))
;; Error: Assert failed: No more than 1024 pending takes are allowed on a single channel.

As the example above shows there’s a limit of pending puts or takes on a channel, it’s currently 1024 but that is an implementation detail that may change. Note that there can’t be both pending puts and pending takes on a channel since puts will immediately succeed if there are pending takes and viceversa.

Channels support buffering of put operations. If we create a channel with a buffer the put operations will succeed immediately if there’s room in the buffer and be enqueued otherwise. Let’s illustrate the point creating a channel with a buffer of one element. The chan constructors accepts a number as its first argument which will cause it to have a buffer with the given size:

(require '[cljs.core.async :refer [chan put! take!]])

(def ch (chan 1))

(put! ch 42 #(println "Put succeeded!"))
;; Put succeeded!
;; => true

(dotimes [n 1024]
  (put! ch n))
;; => nil

(put! ch 42)
;; Error: Assert failed: No more than 1024 pending puts are allowed on a single channel.

What happened in the example above? We created a channel with a buffer of size 1 and performed a put operation on it that succeeded immediately because the value was buffered. After that we did another 1024 puts to fill the pending put queue and, when trying to put one value more the channel complained about not being able to enqueue more puts.

Now that we know about how channels work and what are buffers used for let’s explore the different buffers that core.async implements. Different buffers have different policies and it’s interesting to know all of them to know when to use what. Channels are unbuffered by default.

Fixed

The fixed size buffer is the one that is created when we give the chan constructor a number and it will have the size specified by the given number. It is the simplest possible buffer: when full, puts will be enqueued.

The chan constructor accepts either a number or a buffer as its first argument. The two channels created in the following example both use a fixed buffer of size 32:

(require '[cljs.core.async :refer [chan buffer]])

(def a-ch (chan 32))

(def another-ch (chan (buffer 32)))
Dropping

The fixed buffer allows put operations to be enqueued. However, as we saw before, puts are still queued when the fixed buffer is full. If we wan’t to discard the put operations that happen when the buffer is full we can use a dropping buffer.

Dropping buffers have a fixed size and, when they are full puts will complete but their value will be discarded. Let’s illustrate the point with an example:

(require '[cljs.core.async :refer [chan dropping-buffer put! take!]])

(def ch (chan (dropping-buffer 2)))

(put! ch 40)
;; => true
(put! ch 41)
;; => true
(put! ch 42)
;; => true

(take! ch #(println "Got" %))
;; Got 40
;; => nil
(take! ch #(println "Got" %))
;; Got 41
;; => nil
(take! ch #(println "Got" %))
;; => nil

We performed three put operations and the three of them succeded but, since the dropping buffer of the channel has size 2, only the first two values were delivered to the takers. As you can observe the third take is enqueued since there is no value available, the third put’s value (42) was discarded.

Sliding

The sliding buffer has the opposite policy than the dropping buffer. When full puts will complete and the oldest value will be discarded in favor of the new one. The sliding buffer is useful when we are interested in processing the last puts only and we can afford discarding old values.

(require '[cljs.core.async :refer [chan sliding-buffer put! take!]])

(def ch (chan (sliding-buffer 2)))

(put! ch 40)
;; => true
(put! ch 41)
;; => true
(put! ch 42)
;; => true

(take! ch #(println "Got" %))
;; Got 41
;; => nil
(take! ch #(println "Got" %))
;; Got 42
;; => nil
(take! ch #(println "Got" %))
;; => nil

We performed three put operations and the three of them succeded but, since the sliding buffer of the channel has size 2, only the last two values were delivered to the takers. As you can observe the third take is enqueued since there is no value available since the first put’s value was discarded.

Transducers

As mentioned in the section about transducers, putting values in a channel can be thought as a transducible process. This means that we can create channels and hand them a transducer, giving us the ability to transform the input values before being put in the channel.

If we want to use a transducer with a channel we must supply a buffer since the reducing function that will be modified by the transducer will be the buffer’s add function. A buffer’s add function is a reducing function since it takes a buffer and an input and returns a buffer with such input incorporated.

(require '[cljs.core.async :refer [chan put! take!]])

(def ch (chan 1 (map inc)))

(put! ch 41)
;; => true

(take! ch #(println "Got" %))
;; Got 42
;; => nil

You may be wondering what happens to a channel when the reducing function returns a reduced value. It turns out that the notion of termination for channels is being closed, so channels will be closed when a reduced value is encountered:

(require '[cljs.core.async :refer [chan put! take!]])

(def ch (chan 1 (take 2)))

(take! ch #(println "Got" %))
;; => nil
(take! ch #(println "Got" %))
;; => nil
(take! ch #(println "Got" %))
;; => nil

(put! ch 41)
;; => true
(put! ch 42)
;; Got 41
;; => true
(put! ch 43)
;; Got 42
;; Got nil
;; => false

We used the take stateful transducer to allow maximum 2 puts into the channel. We then performed three take operations on the channel and we expect only two to receive a value. As you can see in the above example the third take got the sentinel nil value which indicates that the channel was closed. Also, the third put operation returned false indicating that it didn’t take place.

Handling exceptions

If adding a value to a buffer throws an exception core.async the operation will fail and the exception will be logged to the console. However, channel constructors accept a third argument: a function for handling exceptions.

When creating a channel with an exception handler it will be called with the exception whenever an exception occurs. If the handler returns nil the operation will fail silently and if it returns another value the add operation will be retried with such value.

(require '[cljs.core.async :refer [chan put! take!]])

(enable-console-print!)

(defn exception-xform
  [rfn]
  (fn [acc input]
    (throw (js/Error. "I fail!"))))

(defn handle-exception
  [ex]
  (println "Exception message:" (.-message ex))
  42)

(def ch (chan 1 exception-xform handle-exception))

(put! ch 0)
;; Exception message: I fail!
;; => true

(take! ch #(println "Got:" %))
;; Got: 42
;; => nil
Offer and Poll

We’ve learned about the two basic operations on channels so far: put! and take!. They either take or put a value and are enqueued if they can’t be performed immediately. Both functions are asynchronous because of their nature: they can succeed but be completed at a later time.

core.async has two synchronous operations for putting or taking values: offer! and poll!. Let’s see how they work through examples.

offer! puts a value in a channel if it’s possible to do so immediately. It returns true if the channel received the value and false otherwise. Note that, unlike with put!, offer! cannot distinguish between closed and open channels.

(require '[cljs.core.async :refer [chan offer!]])

(def ch (chan 1))

(offer! ch 42)
;; => true

(offer! ch 43)
;; => false

poll! takes a value from a channel if it’s possible to do so immediately. Returns the value if succesful and nil otherwise. Unlike take!, poll! cannot distinguish closed and open channels.

(require '[cljs.core.async :refer [chan offer! poll!]])

(def ch (chan 1))

(poll! ch)
;; => nil

(offer! ch 42)
;; => true

(poll! ch)
;; => 42

5.6.2. Processes

We learned all about channels but there is still a missing piece in the puzzle: processes. Processes are pieces of logic that run independently and use channels for communication and coordination. Puts and takes inside a process will stop the process until the operation completes. Stopping a process doesn’t block the only thread we have in the environments where ClojureScript runs. Instead, it will be resumed at a later time when the operation is waiting for being performed.

Processes are launched using the go macro and puts and takes use the <! and >! placeholders. The go macro rewrites your code to use callbacks but inside go everything looks like synchronous code, which makes understanding it straightforward:

(require '[cljs.core.async :refer [chan <! >!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(enable-console-print!)

(def ch (chan))

(go
  (println [:a] "Gonna take from channel")
  (println [:a] "Got" (<! ch)))

(go
  (println [:b] "Gonna put on channel")
  (>! ch 42)
  (println [:b] "Just put 42"))

;; [:a] Gonna take from channel
;; [:b] Gonna put on channel
;; [:b] Just put 42
;; [:a] Got 42

In the above example we are launching a process with go that takes a value from ch and prints it to the console. Since the value isn’t immediately available it will park until it can resume. After that we launch another process that puts a value on the channel.

Since there is a pending take the put operation succeeds and the value is delivered to the first process, then both processes terminate.

Both go blocks run independently and, even though they are executed asynchronously, they look like synchronous code. The above go blocks are fairly simple but being able to write concurrent processes that coordinate via channels is a very powerful tool for implementing complex asynchronous workflows. Channels also offer a great decoupling of producers and consumers.

Processes can wait for an arbitrary amount of time too, there is a timeout function that return a channel that will be closed after the given amount of miliseconds. Combining a timeout channel with a take operation inside a go block gives us the ability to sleep:

(require '[cljs.core.async :refer [<! timeout]])
(require-macros '[cljs.core.async.macros :refer [go]])

(enable-console-print!)

(defn seconds
  []
  (.getSeconds (js/Date.)))

(println "Launching go block")

(go
  (println [:a] "Gonna take a nap" (seconds))
  (<! (timeout 1000))
  (println [:a] "I slept one second, bye!" (seconds)))

(println "Block launched")

;; Launching go block
;; Block launched
;; [:a] Gonna take a nap 9
;; [:a] I slept one second, bye! 10

As we can see in the messages printed, the process does nothing for one second when it blocks in the take operation of the timeout channel. The program continues and after one second the process resumes and terminates.

Choice

Apart from putting and taking one value at a time inside a go block we can also make a non-deterministic choice on multiple channel operations using alts!. alts! is given a series of channel put or take operations (note that we can also try to put and take in a channel at the same time) and only performs one as soon as is ready; if multiple operations can be performed when calling alts! it will do a pseudo random choice by default.

We can easily try an operation on a channel and cancel it after a certain amount of time combining the timeout function and alts!. Let’s see how:

(require '[cljs.core.async :refer [chan <! timeout alts!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(enable-console-print!)

(def ch (chan))

(go
  (println [:a] "Gonna take a nap")
  (<! (timeout 1000))
  (println [:a] "I slept one second, trying to put a value on channel")
  (>! ch 42)
  (println [:a] "I'm done!"))

(go
  (println [:b] "Gonna try taking from channel")
  (let [cancel (timeout 300)
        [value ch] (alts! [ch cancel])]
    (if (= ch cancel)
      (println [:b] "Too slow, take from channel cancelled")
      (println [:b] "Got" value))))

;; [:a] Gonna take a nap
;; [:b] Gonna try taking from channel
;; [:b] Too slow, take from channel cancelled
;; [:a] I slept one second, trying to put a value on channel

In the example above we launched a go block that, after waiting for a second, puts a value in the ch channel. The other go block creates a cancel channel, which will be closed after 300 miliseconds. After that, it tries to read from both ch and cancel at the same time using alts!, which will succeed whenever it can take a value from either of those channels. Since cancel is closed after 300 miliseconds, alts! will succeed since takes from closed channel return the nil sentinel. Note that alts! returns a two-element vector with the returned value of the operation and the channel where it was performed.

This is why we are able to detect whether the read operation was performed in the cancel channel or in ch. I suggest you copy this example and set the first process timeout to 100 miliseconds to see how the read operation on ch succeeds.

We’ve learned how to choose between read operations so let’s look at how to express a conditional write operation in alts!. Since we need to provide the channel and a value to try to put on it, we’ll use a two element vector with the channel and the value for representing write operations.

Let’s see an example:

(require '[cljs.core.async :refer [chan <! alts!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(enable-console-print!)

(def a-ch (chan))
(def another-ch (chan))

(go
  (println [:a] "Take a value from `a-ch`")
  (println [:a] "Got" (<! a-ch))
  (println [:a] "I'm done!"))

(go
  (println [:b] "Take a value from `another-ch`")
  (println [:a] "Got" (<! another-ch))
  (println [:b] "I'm done!"))

(go
  (println [:c] "Gonna try putting in both channels simultaneously")
  (let [[value ch] (alts! [[a-ch 42]
                           [another-ch 99]])]
    (if (= ch a-ch)
      (println [:c] "Put a value in `a-ch`")
      (println [:c] "Put a value in `another-ch`"))))

;; [:a] Take a value from `a-ch`
;; [:b] Take a value from `another-ch`
;; [:c] Gonna try putting in both channels simultaneously
;; [:c] Put a value in `a-ch`
;; [:a] Got 42
;; [:a] I'm done!

When running the above example only the put operation on the a-ch channel has succeeded. Since both channels are ready to take a value when the alts! occurs you may get different results when running this code.

Priority

alts! default is to make a non-deterministic choice whenever several operations are ready to be performed. We can instead give priority to the operations passing the :priority option to alts!. Whenever :priority is true, if more than one operation is ready they will be tried in order.

(require '[cljs.core.async :refer [chan >! alts!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(enable-console-print!)

(def a-ch (chan))
(def another-ch (chan))

(go
  (println [:a] "Put a value on `a-ch`")
  (>! a-ch 42)
  (println [:a] "I'm done!"))

(go
  (println [:b] "Put a value on `another-ch`")
  (>! another-ch 99)
  (println [:b] "I'm done!"))

(go
  (println [:c] "Gonna try taking from both channels with priority")
  (let [[value ch] (alts! [a-ch another-ch] :priority true)]
    (if (= ch a-ch)
      (println [:c] "Got" value "from `a-ch`")
      (println [:c] "Got" value "from `another-ch`"))))

;; [:a] Put a value on `a-ch`
;; [:a] I'm done!
;; [:b] Put a value on `another-ch`
;; [:b] I'm done!
;; [:c] Gonna try taking from both channels with priority
;; [:c] Got 42 from `a-ch`

Since both a-ch and another-ch had a value to read when the alts! was executed and we set the :priority option to true, a-ch has preference. You can try deleting the :priority option and running the example multiple times to see that, without priority, alts! makes a non-deterministic choice.

Defaults

Another interesting bit of alts! is that it can return immediately if no operation is ready and we provide a default value. We can conditionally do a choice on the operations if and only if any of them is ready, returning a default value if it’s not.

(require '[cljs.core.async :refer [chan alts!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(def a-ch (chan))
(def another-ch (chan))

(go
  (println [:a] "Gonna try taking from any of the channels without blocking")
  (let [[value ch] (alts! [a-ch another-ch] :default :not-ready)]
    (if (and (= value :not-ready)
             (= ch :default))
      (println [:a] "No operation is ready, aborting")
      (println [:a] "Got" value))))

;; [:a] Gonna try taking from any of the channels without blocking
;; [:a] No operation is ready, aborting

As you can see in the above example, if no operation is ready the value returned by alts! is the one we supplied after the :default key when calling it and the channel is the :default keyword itself.

5.6.3. Combinators

Now that we’re acquainted with channels and processes it’s time to explore some interesting combinators for working with channels that are present in core.async. This section includes a brief description of all of them together with a simple example of their usage.

pipe

pipe takes an input and output channels and pipes all the values put on the input channel to the output one. The output channel is closed whenever the source is closed unless we provide a false third argument:

(require '[cljs.core.async :refer [chan pipe put! <! close!]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def in (chan))
(def out (chan))

(pipe in out)

(go-loop [value (<! out)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (println [:a] "Waiting for a value")
      (recur (<! out)))))

(put! in 0)
;; => true
(put! in 1)
;; => true
(close! in)

;; [:a] Got 0
;; [:a] Waiting for a value
;; [:a] Got 1
;; [:a] Waiting for a value
;; [:a] I'm done!

In the above example we used the go-loop macro for reading values recursively until the out channel is closed. Notice that when we close the in channel the out channel is closed too, making the go-loop terminate.

pipeline-async

pipeline-async takes a number for controlling parallelism, an output channel, an asynchronous function and an input channel. The asynchronous function has two arguments: the value put in the input channel and a channel where it should put the result of its asynchronous operation, closing the result channel after finishing. The number controls the number of concurrent go blocks that will be used for calling the asynchronous function with the inputs.

The output channel will receive outputs in an order relative to the input channel, regardless the time each asynchronous function call takes to complete. It has an optional last parameter that controls whether the output channel will be closed when the input channel is closed, which defaults to true.

(require '[cljs.core.async :refer [chan pipeline-async put! <! close!]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def in (chan))
(def out (chan))
(def parallelism 3)

(defn wait-and-put [value ch]
  (let [wait (rand-int 1000)]
    (js/setTimeout (fn []
                     (println "Waiting" wait "miliseconds for value" value)
                     (put! ch wait)
                     (close! ch))
                   wait)))

(pipeline-async parallelism out wait-and-put in)

(go-loop [value (<! out)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (println [:a] "Waiting for a value")
      (recur (<! out)))))

(put! in 1)
(put! in 2)
(put! in 3)
(close! in)

;; Waiting 164 miliseconds for value 3
;; Waiting 304 miliseconds for value 2
;; Waiting 908 miliseconds for value 1
;; [:a] Got 908
;; [:a] Waiting for a value
;; [:a] Got 304
;; [:a] Waiting for a value
;; [:a] Got 164
;; [:a] Waiting for a value
;; [:a] I'm done!
pipeline

pipeline is similar to pipeline-async but instead of taking and asynchronous function it takes a transducer instead. The transducer will be applied independently to each input.

(require '[cljs.core.async :refer [chan pipeline put! <! close!]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def in (chan))
(def out (chan))
(def parallelism 3)

(pipeline parallelism out (map inc) in)

(go-loop [value (<! out)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (println [:a] "Waiting for a value")
      (recur (<! out)))))

(put! in 1)
(put! in 2)
(put! in 3)
(close! in)

;; [:a] Got 2
;; [:a] Waiting for a value
;; [:a] Got 3
;; [:a] Waiting for a value
;; [:a] Got 4
;; [:a] Waiting for a value
;; [:a] I'm done!
split

split takes a predicate and a channel and returns a vector with two channels, the first of which will receive the values for which the predicate is true, the second will receive those for which the predicate is false. We can optionally pass a buffer or number for the channels with the third (true channel) and fourth (false channel) arguments.

(require '[cljs.core.async :refer [chan split put! <! close!]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def in (chan))
(def chans (split even? in))
(def even-ch (first chans))
(def odd-ch (second chans))

(go-loop [value (<! even-ch)]
  (if (nil? value)
    (println [:evens] "I'm done!")
    (do
      (println [:evens] "Got" value)
      (println [:evens] "Waiting for a value")
      (recur (<! even-ch)))))

(go-loop [value (<! odd-ch)]
  (if (nil? value)
    (println [:odds] "I'm done!")
    (do
      (println [:odds] "Got" value)
      (println [:odds] "Waiting for a value")
      (recur (<! odd-ch)))))

(put! in 0)
(put! in 1)
(put! in 2)
(put! in 3)
(close! in)

;; [:evens] Got 0
;; [:evens] Waiting for a value
;; [:odds] Got 1
;; [:odds] Waiting for a value
;; [:odds] Got 3
;; [:odds] Waiting for a value
;; [:evens] Got 2
;; [:evens] Waiting for a value
;; [:evens] I'm done!
;; [:odds] I'm done!
reduce

reduce takes a reducing function, initial value and an input channel. It returns a channel with the result of reducing over all the values put on the input channel before closing it using the given initial value as the seed.

(require '[cljs.core.async :as async :refer [chan put! <! close!]])
(require-macros '[cljs.core.async.macros :refer [go]])

(def in (chan))

(go
  (println "Result" (<! (async/reduce + (+) in))))

(put! in 0)
(put! in 1)
(put! in 2)
(put! in 3)
(close! in)

;; Result: 6
onto-chan

onto-chan takes a channel and a collection and puts the contents of the collection into the channel. It closes the channel after finishing although it accepts a third argument for specifying if it should close it or not. Let’s rewrite the reduce example using onto-chan:

(require '[cljs.core.async :as async :refer [chan put! <! close! onto-chan]])
(require-macros '[cljs.core.async.macros :refer [go]])

(def in (chan))

(go
  (println "Result" (<! (async/reduce + (+) in))))

(onto-chan in [0 1 2 3])

;; Result: 6
to-chan

to-chan takes a collection and returns a channel where it will put every value in the collection, closing the channel afterwards.

(require '[cljs.core.async :refer [chan put! <! close! to-chan]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def ch (to-chan (range 3)))

(go-loop [value (<! ch)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (println [:a] "Waiting for a value")
      (recur (<! ch)))))

;; [:a] Got 0
;; [:a] Waiting for a value
;; [:a] Got 1
;; [:a] Waiting for a value
;; [:a] Got 2
;; [:a] Waiting for a value
;; [:a] I'm done!
merge

merge takes a collection of input channels and returns a channel where it will put every value that is put on the input channels. The returned channel will be closed when all the input channels have been closed. The returned channel will be unbuffered by default but a number or buffer can be provided as the last argument.

(require '[cljs.core.async :refer [chan put! <! close! merge]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

(def in1 (chan))
(def in2 (chan))
(def in3 (chan))

(def out (merge [in1 in2 in3]))

(go-loop [value (<! out)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (println [:a] "Waiting for a value")
      (recur (<! out)))))

(put! in1 1)
(close! in1)
(put! in2 2)
(close! in2)
(put! in3 3)
(close! in3)

;; [:a] Got 3
;; [:a] Waiting for a value
;; [:a] Got 2
;; [:a] Waiting for a value
;; [:a] Got 1
;; [:a] Waiting for a value
;; [:a] I'm done!

5.6.4. Higher-level abstractions

We’ve learned the about the low-level primitives of core.async and the combinators that it offers for working with channels. core.async also offers some useful, higher-level abstractions on top of channels that can serve as building blocks for more advanced functionality.

Mult

Whenever we have a channel whose values have to be broadcasted to many others, we can use mult for creating a multiple of the supplied channel. Once we have a mult, we can attach channels to it using tap and dettach them using untap. Mults also support removing all tapped channels at once with untap-all.

Every value put in the source channel of a mult is broadcasted to all the tapped channels, and all of them must accept it before the next item is broadcasted. For preventing slow takers from blocking the mult’s values we must use buffering on the tapped channels judiciously.

Closed tapped channels are removed automatically from the mult. When putting a value in the source channels when there are still no taps such value will be dropped.

(require '[cljs.core.async :refer [chan put! <! close! timeout mult tap]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

;; Source channel and mult
(def in (chan))
(def m-in (mult in))

;; Sink channels
(def a-ch (chan))
(def another-ch (chan))

;; Taker for `a-ch`
(go-loop [value (<! a-ch)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Got" value)
      (recur (<! a-ch)))))

;; Taker for `another-ch`, which sleeps for 3 seconds between takes
(go-loop [value (<! another-ch)]
  (if (nil? value)
    (println [:b] "I'm done!")
    (do
      (println [:b] "Got" value)
      (println [:b] "Resting 3 seconds")
      (<! (timeout 3000))
      (recur (<! another-ch)))))

;; Tap the two channels to the mult
(tap m-in a-ch)
(tap m-in another-ch)

;; See how the values are delivered to `a-ch` and `another-ch`
(put! in 1)
(put! in 2)

;; [:a] Got 1
;; [:b] Got 1
;; [:b] Resting for 3 seconds
;; [:a] Got 2
;; [:b] Got 2
;; [:b] Resting for 3 seconds
Pub-sub

After learning about mults you could imagine how to implement a pub-sub abstraction on top of mult, tap and untap but since it’s a widely used communication mechanism core.async already implements this functionality.

Instead of creating a mult from a source channel, we create a publication with pub giving it a channel and a function that will be used for extracting the topic of the messages.

We can subscribe to a publication with sub, giving it the publication we want to subscribe to, the topic we are interested in and a channel to put the messages that have the given topic. Note that we can subscribe a channel to multiple topics.

unsub can be given a publication, topic and channel for unsubscribing such channel from the topic. unsub-all can be given a publication and a topic to unsubscribe every channel from the given topic.

(require '[cljs.core.async :refer [chan put! <! close! pub sub]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

;; Source channel and publication
(def in (chan))
(def publication (pub in :action))

;; Sink channels
(def a-ch (chan))
(def another-ch (chan))

;; Channel with `:increment` action
(sub publication :increment a-ch)

(go-loop [value (<! a-ch)]
  (if (nil? value)
    (println [:a] "I'm done!")
    (do
      (println [:a] "Increment:" (inc (:value value)))
      (recur (<! a-ch)))))

;; Channel with `:double` action
(sub publication :double another-ch)

(go-loop [value (<! another-ch)]
  (if (nil? value)
    (println [:b] "I'm done!")
    (do
      (println [:b] "Double:" (* 2 (:value value)))
      (recur (<! another-ch)))))

;; See how values are delivered to `a-ch` and `another-ch` depending on their action
(put! in {:action :increment :value 98})
(put! in {:action :double :value 21})

;; [:a] Increment: 99
;; [:b] Double: 42
Mixer

As we learned in the section about core.async combinators, we can use the merge function for combining multiple channels into one. When merging multiple channels, every value put in the input channels will end up in the merged channel. However, we may want more finer-grained control over which values put in the input channels end up in the output channel, that’s where mixers come in handy.

core.async gives us the mixer abstraction, which we can use to combine multiple input channnels into an output channel. The interesting part of the mixer is that we can mute, pause and listen exclusively to certain input channels.

We can create a mixer given an output channel with mix. Once we have a mixer we can add input channels into the mix using admix, remove it using unmix or remove every input channel with unmix-all.

For controlling the state of the input channel we use the toggle function giving it the mixer and a map from channels to their states. Note that we can add channels to the mix using toggle, since the map will be merged with the current state of the mix. The state of a channel is a map which can have the keys :mute, :pause and :solo mapped to a boolean.

Let’s see what muting, pausing and soloing channels means:

  • A muted input channel means that, while still taking values from it, they won’t be forwarded to the output channel. Thus, while a channel is muted, all the values put in it will be discarded.

  • A paused input channel means that no values will be taken from it. This means that values put in the channel won’t be forwarded to the output channel nor discarded.

  • When soloing one or more channels the output channel will only receive the values put in soloed channels. By default non-soloed channels are muted but we can use solo-mode to decide between muting or pausing non-soloed channels.

That was a lot of information so let’s see an example to improve our understanding. First of all, we’ll set up a mixer with an out channel and add three input channels to the mix. After that, we’ll be printing all the values received on the out channel to illustrate the control over input channels:

(require '[cljs.core.async :refer [chan put! <! close! mix admix
                                   unmix toggle solo-mode]])
(require-macros '[cljs.core.async.macros :refer [go-loop]])

;; Output channel and mixer
(def out (chan))
(def mixer (mix out))

;; Input channels
(def in-1 (chan))
(def in-2 (chan))
(def in-3 (chan))

(admix mixer in-1)
(admix mixer in-2)
(admix mixer in-3)

;; Let's listen to the `out` channel and print what we get from it
(go-loop [value (<! out)]
  (if (nil? value)
    (println [:a] "I'm done")
    (do
      (println [:a] "Got" value)
      (recur (<! out)))))

By default, every value put in the input channels will be put in the out channel:

(do
  (put! in-1 1)
  (put! in-2 2)
  (put! in-3 3))

;; [:a] Got 1
;; [:a] Got 2
;; [:a] Got 3

Let’s pause the in-2 channel, put a value in every input channel and resume in-2:

(toggle mixer {in-2 {:pause true}})
;; => true

(do
  (put! in-1 1)
  (put! in-2 2)
  (put! in-3 3))

;; [:a] Got 1
;; [:a] Got 3

(toggle mixer {in-2 {:pause false}})

;; [:a] Got 2

As you can see in the example above, the values put in the paused channels aren’t discarded. For discarding values put in an input channel we have to mute it, let’s see an example:

(toggle mixer {in-2 {:mute true}})
;; => true

(do
  (put! in-1 1)
  (put! in-2 2)  ;; `out` will never get this value since it's discarded
  (put! in-3 3))

;; [:a] Got 1
;; [:a] Got 3

(toggle mixer {in-2 {:mute false}})

We put a value 2 in the in-2 channel and, since the channel was muted at the time, the value is discarded and never put into out. Let’s look at the third state a channel can be inside a mixer: solo.

As we mentioned before, soloing channels of a mixer implies muting the rest of them by default:

(toggle mixer {in-1 {:solo true}
               in-2 {:solo true}})
;; => true

(do
  (put! in-1 1)
  (put! in-2 2)
  (put! in-3 3)) ;; `out` will never get this value since it's discarded

;; [:a] Got 1
;; [:a] Got 2

(toggle mixer {in-1 {:solo false}
               in-2 {:solo false}})

However, we can set the mode the non-soloed channels will be in while there are soloed channels. Let’s set the default non-solo mode to pause instead of the default mute:

(solo-mode mixer :pause)
;; => true
(toggle mixer {in-1 {:solo true}
               in-2 {:solo true}})
;; => true

(do
  (put! in-1 1)
  (put! in-2 2)
  (put! in-3 3))

;; [:a] Got 1
;; [:a] Got 2

(toggle mixer {in-1 {:solo false}
               in-2 {:solo false}})

;; [:a] Got 3

6. Acknowledgments

Special thanks to:

  • J David Eisenberg: For the huge amount of time spend in fixing all kind of errors and writing entire sections of the book, as well as making very valuable suggestions.

And here is an inevitably incomplete list of MUCH-APPRECIATED CONTRIBUTORS — people who have submitted corrections, new ideas and generally made the ClojureScript Unraveled book much better:

  • Anler Hernández Peral (@anler)

  • Diego Sevilla Ruiz (@dsevilla)

  • Eduardo Ferro Aldama (@eferro)

  • Tyler Anderson (@Tyler-Anderson)

  • Chris Ulrich (@chrisulrich)

  • Jean Hadrien Chabran (@jhchabran)

  • Tienson Qin (@tiensonqin)

  • FungusHumungus (@FungusHumungus),

  • Chris Charles (@ccharles)

  • Jearvon Dharrie (@iamjarvo)

  • Shaun LeBron (@shaunlebron)

  • Wodin (@wodin)

  • Crocket (@crocket)

7. Further Reading

Here is a list of more resources about ClojureScript.