Understanding the Clojure Development Ecosystem

Lots of books, talks, and blog posts help people learn Clojure the language, but leave out prosaic issues outside of the language, like development environments and using libraries. This covers the gap between dipping your toe in and confident proficiency.

This guide assumes you are aware of Clojure and want to become a Clojure programmer.

Sections

Why Clojure?
The Philosophy of Clojure
Language Tools
Rich Hickey’s Talks
Baby Steps
Leiningen
nREPL
nrepl.el
Helpful Tips
Appendix – Links

Why Clojure?

Nothing says it better than the front page of clojure.org:

“[Clojure] is designed to be a general-purpose language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming. Clojure is a compiled language – it compiles directly to JVM bytecode, yet remains completely dynamic. Every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.

Clojure is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system. Clojure is predominantly a functional programming language, and features a rich set of immutable, persistent data structures. When mutable state is needed, Clojure offers a software transactional memory system and reactive Agent system that ensure clean, correct, multithreaded designs.” – Rich Hickey, creator of Clojure

The Philosophy of Clojure

Clojure is an opinionated language. Rich Hickey might not be as brash as DHH is with Rails, but his strong opinions are infused throughout the language and community. If you don’t understand the programming philosphy behind Clojure, you will stumble around, fighting the language (and you will lose).

Simplicity – Designs should be broken down into atomic, contained, composable units. This is different than ease, which is a measure of how little effort it takes to solve today’s problems with current resources. Simplicity gives you agility to respond to new opportunities that a complex design would prevent you from pursuing.

Correctness – The more accurate your mental model of your code, the more confidently and accurately you can change it or add to it without breaking it.

Manage Difficulty – With a better language and tools, you can solve harder problems, or solve easier problems faster.

Language Tools

Clojure has many awesome features to achieve those philosophical goals.

  • First class functions – Functions can be assigned to a variable, passed as arguments to other functions, and returned by other functions. The enables new kinds of abstraction and composition.
  • Immutability – Data is expected not to change, outside of some controlled circumstances. Variables are assigned, but not changed – a variable is a name to refer to a fact, not a container for a value. This makes using values more predictable and concurrency friendly. Clojure has lots of academic and implementation work making this run efficiently.
  • JVM Foundation – By compiling to JVM byte code and including native Java interop, Clojure has performance, distribution, and library advantages over other from-scratch languages.
  • Macros – Lisp languages are homoiconic – their code is written in the same data structures as the language uses, so code generation is natural and easy.
  • Datomic¬†– Rich has released a database-ish product called Datomic that embodies the same principles of simplicity, correctness, and immutability

Everyone everywhere ever has written more and better about all these topics. Start with any book on Clojure, preferably Programming Clojure (WARNING: Clojure book titles tend to sound the same)

Rich Hickey’s Talks

You should watch the following talks to understand the base principles and language strategies for achieving them by watching these talks. I would recommend that any software developer watches them, whether or not they ever do anything with Clojure.

Baby Steps

If you want an instant, no-strings-attached way to play with Clojure, there are a few options.

Try Clojure – A web-based console (REPL in Lisp/Clojure terms) that will evaluate any code you type in. It also includes a friendly tutorial to walk you through the basics.

4Clojure – A set of exercises geared around helping you learn Clojure. For each problem, you enter your solution and the site runs unit tests against your code. Once you pass, you can view solutions from other users. I recommend following a few users from the leaderboard to see a variety of solutions for each problem. The site doesn’t teach you Clojure, but you have to learn in order to solve the increasingly difficult challenges.

Himera Рsimilar to Try Clojure, but for ClojureScript. (See also http://clojurescript.net/)

Leiningen

Once you’re past playing in a sandboxed environment, you should *immediately* install Leiningen and use that with your Clojure code. In the words of the excellent documentation page

“Leiningen is for automating Clojure projects without setting your hair on fire.”

NOTE: Make sure to get the Leiningen v2 preview script, don’t start anything new with v1.x. v2 is very stable and has a ton of improvements, it’s waiting on a change to the Java package management system before officially releasing.

I came to Clojure from Ruby on Rails, so here’s the analogy. Leiningen does for Clojure what rvm, rake, bundler, rubygems, and the Rails command-line does for Ruby.

Structurally, leiningen starts with a simple bootstrap script that you download and execute, then it will download and manage its dependencies when you run it. You get a ton of features, including a better REPL, the ability to connect your REPL to local or remote Clojure instances, package management, dependency management, generate project scaffolding, run code, start servers, package and publish code, and more. It also has a plugin architecture so you can use any of the existing plugins or write your own.

I repeat: Any non-trivial Clojure code should be in a Leiningen project!

nREPL

At its most basic level, Clojure is a library loaded into the JVM. If you run this library, you get a very basic REPL. Since this REPL isn’t very good (it lacks command history and cursor movement keyboard shortcuts, for example), people developed better REPL libraries and plugins for different editors.

One of the most commonly used and powerful was SLIME for Emacs. It started a Clojure instance with a server (called Swank) running that would accept and evaluate commands, and a client program (SLIME) in the editor that integrated code with the compiled version in the running instance. It provided lots of goodies, like inline code evaluation, easy inspection of a function’s documentation or source, finding callers and callees for a function, and access to the state of a running server. It had a variety of implementation problems, mostly because of the differences between SLIME’s Common Lisp roots and Clojure.

Aside from the issues with SLIME, other editor-based REPLs did not work together well. A project needed to run servers for each editor that was working on it, or developers using different editors wouldn’t be able to work together. (more backstory here)

These concerns led to the development of nREPL, which is intended to be a common platform for communicating with Clojure instances. It recently went through a major redesign and now it’s solid enough that the maintainers of swank-clojure have deprecated that project in favor of it. (Nothing is wrong or broken with swank-clojure, but it is no longer being developed. If you already have it working and enjoy it, by all means continue).

nREPL was designed to be extensible and composable, so both its transport mechanisms and functionality can be extended. An example of extending the transport mechanism, the drawbridge library lets you communicate with a remote repl over HTTP instead of sockets, which lets you use nREPL in network environments where you would otherwise not be able to. A library like nrepl-middleware lets you add functionality like documentation lookup, fuzzy symbol completion, and more.

In order to take advantage of these features, a client would need to know to call them, but because the clients and servers are decoupled and nREPL aims to be a common server, any client (whether for Emacs, Eclipse, Vim, etc) can use any features, middlewares, or transports that nREPL supports. This also means that different projects will have different nREPL functionality, based on which plugins and versions they are using.

nREPL is a new project so there are not a lot of clients, middlewares, or transports yet. It currently has fewer features than SLIME/swank-clojure, so established developers may not want to switch yet. But it is written and endorsed by some of the most prominent members of the Clojure community and there’s a very bright future ahead of it, so anyone beginning now should use it.

nrepl.el

Given that most Clojure develors use Emacs, it’s unsurprising that Emacs would have a solid nREPL clients – nrepl.el. It is very similar to SLIME, although lacking some features, most notably the selector and debugger.

In code buffers, it provides lots of conveniences like automatically starting a Clojure instance and nREPL server and connecting to it, evaluating different levels of code, expanding macros, namespace management, file loading, documentation helps, navigating to code, symbol completion, and more. In he REPL it provides has goodies like command history (including search by regex), REPL-specific editing commands, parenthesis management, plus the kitchen sink of editing tools available within Emacs.

Anyone using Emacs+nREPL should use this, as well as anyone looking to have editor compatability with the largest number of Clojure developers.

Helpful Tips

  • People are very friendly and helpful in the community.
  • Everything goes smoother if you understand the philosophy[##Link above##]
  • The simplicity principle means there aren’t big frameworks, and they’re not likely to develop. Your code will consist of intelligently combining simple pieces.
  • The core language will grow slowly and wisely and a lot of functionality will come from libraries. You will need to evaluate and choose which one to use for a given need.
  • Most libraries, including documentation, are hosted on Github. But released versions are published in a way that Leiningen can download, install, and manage for you, so you don’t need to clone a library to use it.
  • Lastly, Clojure (the language and the community) expects a lot of you and rewards you handsomely for it.

Appendix – Links

Counterclockwise – An Eclipse plugin for editing Clojure. I’ve never used it or Eclipse, but I hear good things.
Vimclojure – A filetype, syntax and indent plugin for Clojure.
LightTable – A new IDE with a focus on real-time feedback and interactivity to improve understanding of code. Currently in Alpha, being developed by prominent Clojure devs who have received investment funding to work on it. I’ve heard great things about it and it’s much easier to start with than Emacs.
Ritz – a collection of libraries and nREPL+Swank servers for development and debugging. The backend debugger features are very powerful, but presenting that information in a Clojure-y way hasn’t been solved yet.
ClojureWerkz – a set of well written libraries for common problems like currency manipulation, web crawling, URIs, mail, as well as for interfacing with other projects, like MongoDB, Neo4j, Riak, etc.
Clojure Toolbox – A categorised directory of libraries and tools for Clojure
Clojure Doc – A community driven site for documentation on the Clojure language and ecosystem

When you’re done reading and ready to program, use this excellent guide: Getting Started with Clojure

 

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>