Hardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx Engineering
  • Home
  • Expertise
    • Software Engineering
    • Electrical Engineering
    • Chemical Products and Services
    • Biomedical Engineering
    • Mechanical Engineering
    • Production Management
    • Automation
    • Industrial Design
  • Blog
  • About Us
NextPrevious

Clojure: An improved workflow

By dfohl | Software | 0 comment | 4 May, 2015 | 2

Like many beginning Clojure programmers, I started off following Stuart Sierra’s “Reloaded” workflow guide. While it was a great starting point, there were a number of things that I wanted to change.

  1. If the project doesn’t compile then the REPL doesn’t even start (the “reloaded” guide mentions this toward the end of the post).
  2. There isn’t a good separation of “configuration” and “system”.  I wanted a way to specify various configurations, and launch running systems from those.
  3. I wanted a way, when re-launching a system, to choose to either maintain the current configuration, or specify a new configuration to launch.
  4. I wanted to be able to maintain REPL vars for e.g. db, without having to reinstantiate them individually each time I relaunched a new system var.

1. Starting the Clojure REPL

To fix the first problem, I essentially decimated the user namespace. Now it only contains the functions necessary to reload the source files, but nothing to actually use them.

1
2
(ns user
  (:require [ clojure.tools.namespace.repl] :refer :all))

I put the rest of the system-specific code in a new namespace called repl.

2. Separating “system” and “config”

Now, for the second item, we needed a way to pass a config variable into the start function. However, since the built-in clojure.tools.namespace.repl/refresh function assumes that start is parameterless, I had to make some changes to those functions. The changes are fairly simple: within refresh add an :after-args key and call do-refresh with it. Then within do-refresh apply the new args to your after function rather than calling it directly.

1
2
3
4
5
6
7
8
9
10
(defn- do-refresh [scan-fn after-sym args]
  ...
          (if-let [after (ns-resolve *ns* after-sym)]
            (apply after args)
  ...
            result)))))
 
(defn refresh  [& options]
  (let [{:keys [after after-args]} options]
    (do-refresh dir/scan after after-args)))

Now you can add your env parameter to start and invoke that from reload. Thus, here is the full code we need in repl.clj to enable starting and stopping your system based on any desired configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(ns repl
   (:require [my.clojure.tools.namespace.repl :refer :all]
             [myproject.system :as system]))
 
(def system nil)
 
(defn stop
  "Shuts down and destroys the current development system."
  []
  (system/stop system))
 
(defn- start
  "Starts the current development system."
  [env]
  (alter-var-root #'system (constantly (system/start env))))
 
(defn reload
  "Stops the system and relaunches with optional new config"
  [& [env]]
  (let [env (or env (:env system))]
    (stop)
    (refresh :after 'repl/start :after-args [env])))

3. Optional reconfig at relaunch

You can see the above code, in particular line 20, essentially takes care of the third item as well.  It accepts the optional env that you pass in, or otherwise defaults to the :env of the current system, so long as you follow the convention within your myproject.system/start that your resulting system value has a key :env that contains the passed-in config variable.

4. Convenience variables

Now, when starting up the REPL, you’d think we would jump into namespace repl and start hacking from there. However repl.clj contains code that I want to be able to keep in version control, but I don’t want all my hacking code to be in the VCS, so I don’t have to worry about using private keys and such when I’m just doing experimental development. So instead I create a namespace hack that is excluded from the VCS, and add references to the rest of my project there. The namespace declaration looks like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
(ns hack
  (:require
    ; things you'd want in the repl by default
    [ clojure.java.io :as io]
    [ clojure.string :as str]
    [ clojure.pprint :refer [pprint]]
    [ clojure.repl :refer :all]
    [ clojure.test :as test]
    [ repl :refer :all]
 
    ; another .gitignored file with your configs
    [ env]
 
    ; project namespaces that you want to use
    [ myproject.services.database :as db]
    [ myproject.services.facebook :as fb]
    [ myproject.utility.io :as myio]
 
    ; libraries you want to use
    [ clj-time.core :as time]
    [ clj-time.coerce :as timec]
    [ monger.collection :as mc]
    [ monger.operators :refer :all]))

Then all the hacking you do, you put in a comment block so it doesn’t all get executed when you load the file. Notably, at the top of this Clojure block, I add a quick initialization script to create some handy top-level variables after initializing the system, solving issue #4 above. At the end of the day, the block will look like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
(comment
  ; initialization block
  ; just change to e.g. env/prod and re-run to change environs
  (do
    (repl/reload env/dev)
    (def db (-> repl/system :monger :db))
    (def req {:system repl/system}))
 
  ; misc hacking
  (def post (last (mc/find-maps db :posts)))
  (pprint (myproject.controllers.folders/get-folder-json folder req))
  (pprint (db/group-folder-items db folder))
  (pprint (distinct (map #(-> % :image :folder) (mc/find-maps db :uploads)))))

5. Resulting workflow

Now, my Clojure workflow is, launch the REPL, open hack.clj in my editor (I use cursive, but this will work anywhere), and load the flie into the REPL. I decide which env I need to use, and type that into the do block, and execute it. I’ve got my “system”, my DB, etc all connected as top-level variables, and I can start making database calls via the library, or via my own database wrapper code.

By playing around in the hack namespace, I figure out what changes I need to make to the actual project code. I make those changes to the source files, go back to hack.clj pull in the changes by running that top-level do block again and everything is ready for testing it out. If I want to switch to e.g. the production environment to verify everything works there too, then simply change env/dev to env/prod, run the do block again, and you’re on your way.

Clojure, Programming
dfohl

dfohl

More posts by dfohl

Related Post

  • Routing in Nancy from F#

    By dfohl | 0 comment

    Nancy is my web framework of choice for .NET. It is free, lightweight, easy to use, and comes without all the bloat of the more common .NET frameworks. Nancy makes routing in C# simple. IfRead more

  • Introduction to Lorris Toolbox Analyzer

    By dreynolds | 0 comment

    Lorris Toolbox is an open source application for working with embedded systems and micro-controllers which was developed by Vojtěch Boček in the Czech Republic.  This toolbox of tools has a feature packed packet analyzer tool whichRead more

  • An example of what higher-kinded types could make possible in C#

    By dfohl | 3 comments

    If you’re like me, you spend a lot of time in Visual Studio cranking away at C# code.  You follow the updates to the language with each release, learn the new things that are possibleRead more

  • Building Future Engineers and Programmers

    By Dheadley | 0 comment

    “What Most Schools Don’t Teach” – A video on the importance of educating kids about software and engineering was recently released by Code.org, a non-profit organization tasked with growing the educational options and opportunities forRead more

Leave a Comment

Cancel reply

Your email address will not be published. Required fields are marked *

NextPrevious
  • Home
  • Expertise
  • Blog
  • About Us
Sparx Technologies, LLC. dba Sparx Engineering © 2009 - 2020 | All Rights Reserved
  • Home
  • Expertise
    • Software Engineering
    • Electrical Engineering
    • Chemical Products and Services
    • Biomedical Engineering
    • Mechanical Engineering
    • Production Management
    • Automation
    • Industrial Design
  • Blog
  • About Us
Hardware, Software & Product Development | Sparx Engineering