Clojure : loading dependencies at the REPL

Question

I recently learned (thanks to technomancy) that, at the REPL ---

This fails:

user=> (:require [clojure.set :as set])
java.lang.ClassNotFoundException: clojure.set (NO_SOURCE_FILE:24)

Whereas this succeeds :

user=> (require '[clojure.set :as cs]) 
nil

at loading the clojure.set class.

Context: The former line was copied from a namespaced source file.

My primary question is : What is the change we have made, by swapping the : and ' characters, which now allows for success of the latter command ?

My 2nd question is , in general - what are the guidelines for doing things at the REPL --- as compared with doing things in normal clojure source files ? Assume here that we can load our repl from the root of a LEININGEN project, so at least the jars will be available on disk in the dependencies sub directory.

Solution 1

I'll go from high-level down to your particular problem:

How Clojure (or LISPs) Generally Work

REPLs, or Read-Eval-Print Loops are the core of how LISPs are designed:

• The reader converts a stream of characters into data structures (called Reader Forms).
• The evaluator takes collection of reader forms and evaluates them.
• The printer emits the results of the evaluator.

So when you enter text into a REPL, it goes through each of these steps to process your input and return the output to your terminal.

Reader Forms

First some, clojure reader forms. This will be extremely brief, I encourage you to read or watch (part 1, part 2) about it.

A symbol in clojure is form that can represent a particular value (like a variable). Symbols themselves can be pass around as data. They are similar to pointers in c, just without the memory management stuff.

A symbol with a colon in front of it is a keyword. Keywords are like symbols with the exception that a keyword's value are always themselves - similar to strings or numbers. They're identical to Ruby's symbols (which are also prefixed with colons).

A quote in front of a form tells the evaluator to leave the data structure as-is:

user=> (list 1 2)
(1 2)
user=> '(1 2)
(1 2)
user=> (= (list 1 2) '(1 2))
true

Although quoting can apply to more than just lists, it's primarily used for lists because clojure's evaluator will normally execute lists as a function-like invocation. Using the ' is shorthand to the quote macro:

user=> (quote (1 2)) ; same as '(1 2)
(1 2)

Quoting basically specifies data structure to return and not actual code to execute. So you can quote symbols which refers to the symbol.

user=> 'foo ; not defined earlier
foo

And quoting is recursive. So all the data inside are quoted too:

user=> '(foo bar)
(foo bar)

To get the behavior of (foo bar) without quoting, you can eval it:

user=> (eval '(foo bar)) ; Remember, foo and bar weren't defined yet.
CompilerException java.lang.RuntimeException: Unable to resolve symbol: foo in this context, compiling:(NO_SOURCE_PATH:1)
user=> (def foo identity)
#'user/foo
user=> (def bar 1)
#'user/bar
user=> (eval '(foo bar))
1

There's a lot more to quoting, but that's out of this scope.

Requiring

As for require statements, I'm assuming you found the former in the form of:

(ns my.namespace
    (:require [clojure.set :as set]))

ns is a macro that will transform the :require expression into the latter form you described:

(require '[clojure.set :as set])

Along with some namespacing work. The basics are described when asking for the docs of ns in the REPL.

user=> (doc ns)
-------------------------
clojure.core/ns
([name docstring? attr-map? references*])
Macro
  Sets *ns* to the namespace named by name (unevaluated), creating it
  if needed.  references can be zero or more of: (:refer-clojure ...)
  (:require ...) (:use ...) (:import ...) (:load ...) (:gen-class)
  with the syntax of refer-clojure/require/use/import/load/gen-class
  respectively, except the arguments are unevaluated and need not be
  quoted. (:gen-class ...), when supplied, defaults to :name
  corresponding to the ns name, :main true, :impl-ns same as ns, and
  :init-impl-ns true. All options of gen-class are
  supported. The :gen-class directive is ignored when not
  compiling. If :gen-class is not supplied, when compiled only an
  nsname__init.class will be generated. If :refer-clojure is not used, a
  default (refer 'clojure) is used.  Use of ns is preferred to
  individual calls to in-ns/require/use/import:

REPL usage

In general, don't use ns in the REPL, and just use the require and use functions. But in files, use the ns macro to do those stuff.

Solution 2

The difference is that require is a function used for importing code, whereas :require is a keyword.

Remember what happens when you use a keyword as a function:

=> (type :require)
clojure.lang.Keyword
=> (:require {:abc 1 :require 14})
14

it looks itself up in the map. So when you pass [clojure.set :as set] to a keyword, it's trying to evaluate that to a vector, and fails because it doesn't know what clojure.set is. The Clojure docs say:

Keywords implement IFn for invoke() of one argument (a map) with an optional second argument (a default value). For example (:mykey my-hash-map :none) means the same as (get my-hash-map :mykey :none).

You may have been confused by the ns macro:

(ns foo.bar
  (:refer-clojure :exclude [ancestors printf])
  (:require (clojure.contrib sql sql.tests))    ;; here's :require!
  (:use (my.lib this that))
  (:import (java.util Date Timer Random)
           (java.sql Connection Statement)))

Solution 3

ns macro:

When you type:

(ns some-great-ns 
  :require my-form) 

you use the :require reference in which you state what would you like to use from the given namespace. It is equivalent to writing:

(in-ns 'some-great-ns)
(require 'my-form)

Notice that in the ns form (unlike the in-ns function call), you don’t have to quote your symbol with '. You never have to quote symbols within ns.

require function

As stated, can run: (require 'some-great-ns) in some given namespace so you could use it. To use it, you'll have to use full qualified name, unless you also use: refer function: (refer 'some-great-ns) right after you required the namespace.

You can do those both functions in one: (use 'some-great-ns). Now you don't need to write: (some-great-ns/my-form). Simply: my-form.

And of course you can also use the :as, :exclude, :only and :rename keywords in both the macro reference and in the function.

Differences between the macro and the function:

1. As stated above, usage of symbols in function, no need in the macro
2. You can require multiple libraries in a (:require) reference as follows:

(ns my-great-namespace.core
  (:require [some-other-ns.a.b :as ab]
            [some-other-other-ns.c.d :as cd]))

Where in function writing you should write 2 lines:

(in-ns my-great-namespace.core)
(require 'some-other-ns.a.b :as 'ab)
(require 'some-other-other=ns.c.d :as 'cd)

3. The require reference also allows you to refer names, for example:

(ns my-great-namespace.core
  (:require [some-other-ns.a.b :refer [some-func]]))

Where in function you should do:

(in-ns my-great-namespace.core)
(require 'some-other-ns.a.b)
(refer 'some-other-ns.a.b :only ['some-func])

Comments

[jayunit100]

I guess this realy asks the question of "why require is a keyword in a normal clj file --- but a standard function at the REPL".

[Matt Fenwick]

I think the second part is a bit general and should be removed or split into a totally separate post.

[T.W.R. Cole]

Yes it seems that the OP looked at usage of :require and created mental equivalency to (require), which is understandable; and it probably came from looking at code that used the former; the difference of course being that the forms are (ns ...) vs (require ...)

[Jared Smith]

Maybe I'm just missing it, but what does this add over Jeff's highly-upvoted answer from almost a decade ago?.

[Aaron_ab]

@JaredSmith - Added some comparison between the macro and the function to point the difference between them.