[2012-May-14] Update: Fixed shebang for more Unix variants
What would it take to write scripts in Clojure (and run them on a terminal) the way we write them in Python, Ruby or Groovy? To pipe scripts together to accomplish bigger tasks? What would it take to throw in argument handling and transitive dependency management capabilities to such scripts? This post attempts to answer some of these questions while illustrating how to use the Leiningen plugin lein-exec to write Clojure scripts.
In particular, I assume you have Leiningen 2 and lein-exec 0.2.0 installed. The shebang feature may require that you use a *nix environment.
Command-line eval
Let us begin with evaluating tiny Clojure S-expressions on the command line.
Since the plugin can also evaluate the content from STDIN, we can also do the same thing as above by piping the Clojure code to the plugin:
While clojure.core vars are available, vars from other namespaces may need an explicit reference:
You would need similar quoting when writing the Clojure scripts due to the fact that S-expressions and scripts are simply evaluated by lein-exec.
Scripting
So, let's start with the first script as a file called fib10.clj that simply prints out first 10 numbers in the Fibonacci series:
We can run it using lein-exec:
OK, that is fine and dandy but how do we run fib10.clj as an executable? Here's how – you edit the file and put a shebang on the first line:
Now you can run the fib10.clj as an executable:
Handling arguments
Scripts often need to work with command-line arguments. While using lein-exec arguments are always passed as string via the clojure.core/*command-line-args* dynamic var – first argument is the name of the script followed by the arguments to the script. Below is an example that takes name and scores and prints out name and average:
Upon running the script we can inspect how it responds to different input arguments:
Pipeline
Pipeline is a very powerful concept that enables us to chain simple tasks using their STDIN and STDOUT to build more sophisticated tasks. As we would see below, it is entirely possible to build pipe together scripts written in Clojure by simply reading input from STDIN and writing result to STDOUT:
This script lcase.clj converts the input fed via STDIN to lowercase. Let us see it in action:
Transitive dependencies
Scripting with Clojure is fun! It would only be nicer to have the power of transitive dependency management baked into it. Well, pomegranate that is part of Leiningen 2 already makes that possible. lein-exec wraps pomegranate API to provide its own simpler API. The example below shows how to create a demo web service using lein-exec's deps function and Ring:
The deps function above from leiningen.exec namespace, which is in the lein-exec plugin itself. Here we pull the dpendency ring 1.0.1 which depends on a number of components under it. All those dependencies are pulled in as you run this script. Upon running something this is what you see:
Depending on whether ring is already downloaded to your local Maven repo, you may see the script downloading them first if they don't already exist.
Working in project context
At times you may want to run a script in the context of a project, so that the code being evaluated has access to the project CLASSPATH and other project resources. All of the above usages of lein-exec support an additional switch -p that does the same thing in project scope.
The examples above are applicable when you run them in a project. If you use -p outside of a project it will complain about missing project!
Caveats
While the ability to script in Clojure is fascinating with all the shebang operator, pipeline and dependency management, it's important to know its limitations:
Few of these limitations may be addressed in future versions.
I think many Clojure beginners may see scripting as a good way to learn the language and explore Clojure libraries. At the same time, some of the Clojure app/tool projects may find it easy to distribute the app via a script instead of a full blown JAR with instructions on how to run it. Whatever is your impression, feedback and ideas, please share it in comments. You may like to follow me on Twitter: @kumarshantanu
Happy scripting with Clojure!
What would it take to write scripts in Clojure (and run them on a terminal) the way we write them in Python, Ruby or Groovy? To pipe scripts together to accomplish bigger tasks? What would it take to throw in argument handling and transitive dependency management capabilities to such scripts? This post attempts to answer some of these questions while illustrating how to use the Leiningen plugin lein-exec to write Clojure scripts.
In particular, I assume you have Leiningen 2 and lein-exec 0.2.0 installed. The shebang feature may require that you use a *nix environment.
Command-line eval
Let us begin with evaluating tiny Clojure S-expressions on the command line.
$ lein2 exec -e "(println \"40 and 50 make\" (+ 40 50))" 40 and 50 make 90
Since the plugin can also evaluate the content from STDIN, we can also do the same thing as above by piping the Clojure code to the plugin:
$ echo "(println \"40 and 50 make\" (+ 40 50))" | lein2 exec 40 and 50 make 90
While clojure.core vars are available, vars from other namespaces may need an explicit reference:
$ lein2 exec -e "(use 'clojure.pprint) (pprint (map inc (range 10)))" (1 2 3 4 5 6 7 8 9 10)
You would need similar quoting when writing the Clojure scripts due to the fact that S-expressions and scripts are simply evaluated by lein-exec.
Scripting
So, let's start with the first script as a file called fib10.clj that simply prints out first 10 numbers in the Fibonacci series:
;; Taken from http://j.mp/IiT8UK
(def fib-seq
((fn rfib [a b]
(lazy-seq (cons a (rfib b (+ a b)))))
0 1))
(println (take 10 fib-seq))
We can run it using lein-exec:
$ lein2 exec fib10.clj (0 1 1 2 3 5 8 13 21 34)
OK, that is fine and dandy but how do we run fib10.clj as an executable? Here's how – you edit the file and put a shebang on the first line:
#!/bin/bash lein-exec
;; Taken from http://j.mp/IiT8UK
(def fib-seq
((fn rfib [a b]
(lazy-seq (cons a (rfib b (+ a b)))))
0 1))
(println (take 10 fib-seq))
Now you can run the fib10.clj as an executable:
$ chmod a+x fib10.clj $ ./fib10.clj (0 1 1 2 3 5 8 13 21 34)
Handling arguments
Scripts often need to work with command-line arguments. While using lein-exec arguments are always passed as string via the clojure.core/*command-line-args* dynamic var – first argument is the name of the script followed by the arguments to the script. Below is an example that takes name and scores and prints out name and average:
#!/bin/bash lein-exec
(defn err-println "println for STDERR"
[& args]
(binding [*out* *err*]
(apply println args)))
(defn parse-int "Parse string as an integer. Abort if invalid."
[n]
(try (Integer/parseInt n)
(catch NumberFormatException e
(err-println (str \' n \')
"is not a valid integer")
(System/exit 1))))
(defn avg "Given a sequence of numbers return their average."
[nseq]
(double (/ (apply + nseq) (count nseq))))
(if (>= (count *command-line-args*) 3)
(println (second *command-line-args*)
(avg (map parse-int (drop 2 *command-line-args*))))
(do (err-println "Usage:" (first *command-line-args*)
"name score [score2 ..]")
(System/exit 1)))
Upon running the script we can inspect how it responds to different input arguments:
$ chmod a+x ./avg.clj $ ./avg.clj Usage: ./avg.clj name score [score2 ..] $ ./avg.clj "Nick Foster" Usage: ./avg.clj name score [score2 ..] $ ./avg.clj "Nick Foster" 45 78 65 Nick Foster 62.66666666666667 $ ./avg.clj "Nick Foster" notnum 45 78 65 'notnum' is not a valid integer
Pipeline
Pipeline is a very powerful concept that enables us to chain simple tasks using their STDIN and STDOUT to build more sophisticated tasks. As we would see below, it is entirely possible to build pipe together scripts written in Clojure by simply reading input from STDIN and writing result to STDOUT:
#!/bin/bash lein-exec
(loop []
(let [c (.read *in*)]
(when (>= c 0)
(if (and (>= c (int \A)) (<= c (int \Z)))
(print (Character/toLowerCase (char c)))
(print (char c)))
(recur))))
This script lcase.clj converts the input fed via STDIN to lowercase. Let us see it in action:
$ chmod a+x ./lcase.clj $ echo "5 Quick brown foxes Jumped over 7 lazy dogs" | ./lcase.clj 5 quick brown foxes jumped over 7 lazy dogs $ tree | ./lcase.clj ..result omitted.. $ ls -l ~ | ./lcase.clj ..result omitted..
Transitive dependencies
Scripting with Clojure is fun! It would only be nicer to have the power of transitive dependency management baked into it. Well, pomegranate that is part of Leiningen 2 already makes that possible. lein-exec wraps pomegranate API to provide its own simpler API. The example below shows how to create a demo web service using lein-exec's deps function and Ring:
#!/bin/bash lein-exec
(use '[leiningen.exec :only (deps)])
(deps '[[ring "1.0.1"]])
(defn handler
[request]
{:status 200
:headers {}
:body "Hello from Ring!"})
(use 'ring.adapter.jetty)
(run-jetty handler {:port 3000})
The deps function above from leiningen.exec namespace, which is in the lein-exec plugin itself. Here we pull the dpendency ring 1.0.1 which depends on a number of components under it. All those dependencies are pulled in as you run this script. Upon running something this is what you see:
$ chmod a+x ring.clj $ ./ring.clj 2012-04-22 17:55:35.814:INFO::Logging to STDERR via org.mortbay.log.StdErrLog 2012-04-22 17:55:35.815:INFO::jetty-6.1.25 2012-04-22 17:55:35.824:INFO::Started SocketConnector@0.0.0.0:3000
Depending on whether ring is already downloaded to your local Maven repo, you may see the script downloading them first if they don't already exist.
Working in project context
At times you may want to run a script in the context of a project, so that the code being evaluated has access to the project CLASSPATH and other project resources. All of the above usages of lein-exec support an additional switch -p that does the same thing in project scope.
$ lein2 exec -ep "(use 'foo.bar) (baz :qux)" ..result omitted.. $ cat foo.clj | lein2 exec -p .. result omitted.. $ lein2 exec -p foo.clj .. result omitted..
The examples above are applicable when you run them in a project. If you use -p outside of a project it will complain about missing project!
Caveats
While the ability to script in Clojure is fascinating with all the shebang operator, pipeline and dependency management, it's important to know its limitations:
- As widely known, JVM startup time is a pain even though Leiningen works hard to reduce it.
- Eval'd code runs in the same JVM that runs Leiningen. There's no easy way to customize that.
- Currently, the deps function pulls in dependencies only from Maven Central and Clojars.
- When you run lein-exec in project context, the project map is not accessible to the eval'd code.
Few of these limitations may be addressed in future versions.
I think many Clojure beginners may see scripting as a good way to learn the language and explore Clojure libraries. At the same time, some of the Clojure app/tool projects may find it easy to distribute the app via a script instead of a full blown JAR with instructions on how to run it. Whatever is your impression, feedback and ideas, please share it in comments. You may like to follow me on Twitter: @kumarshantanu
Happy scripting with Clojure!
