Hardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx EngineeringHardware, Software & Product Development | Sparx Engineering
  • Home
  • Expertise
    • Software Engineering
    • Electrical Engineering
    • Chemical Products and Services
    • Biomedical Engineering
    • Mechanical Engineering
    • Production Management
    • Automation
    • Industrial Design
  • Blog
  • Careers
  • About Us
NextPrevious

Clojure: An improved workflow

By dfohl | Software | 0 comment | 4 May, 2015 | 2

Like many beginning Clojure programmers, I started off following Stuart Sierra’s “Reloaded” workflow guide. While it was a great starting point, there were a number of things that I wanted to change.

  1. If the project doesn’t compile then the REPL doesn’t even start (the “reloaded” guide mentions this toward the end of the post).
  2. There isn’t a good separation of “configuration” and “system”.  I wanted a way to specify various configurations, and launch running systems from those.
  3. I wanted a way, when re-launching a system, to choose to either maintain the current configuration, or specify a new configuration to launch.
  4. I wanted to be able to maintain REPL vars for e.g. db, without having to reinstantiate them individually each time I relaunched a new system var.

1. Starting the Clojure REPL

To fix the first problem, I essentially decimated the user namespace. Now it only contains the functions necessary to reload the source files, but nothing to actually use them.

1
2
(ns user
  (:require [ clojure.tools.namespace.repl] :refer :all))

I put the rest of the system-specific code in a new namespace called repl.

2. Separating “system” and “config”

Now, for the second item, we needed a way to pass a config variable into the start function. However, since the built-in clojure.tools.namespace.repl/refresh function assumes that start is parameterless, I had to make some changes to those functions. The changes are fairly simple: within refresh add an :after-args key and call do-refresh with it. Then within do-refresh apply the new args to your after function rather than calling it directly.

1
2
3
4
5
6
7
8
9
10
(defn- do-refresh [scan-fn after-sym args]
  ...
          (if-let [after (ns-resolve *ns* after-sym)]
            (apply after args)
  ...
            result)))))
 
(defn refresh  [& options]
  (let [{:keys [after after-args]} options]
    (do-refresh dir/scan after after-args)))

Now you can add your env parameter to start and invoke that from reload. Thus, here is the full code we need in repl.clj to enable starting and stopping your system based on any desired configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(ns repl
   (:require [my.clojure.tools.namespace.repl :refer :all]
             [myproject.system :as system]))
 
(def system nil)
 
(defn stop
  "Shuts down and destroys the current development system."
  []
  (system/stop system))
 
(defn- start
  "Starts the current development system."
  [env]
  (alter-var-root #'system (constantly (system/start env))))
 
(defn reload
  "Stops the system and relaunches with optional new config"
  [& [env]]
  (let [env (or env (:env system))]
    (stop)
    (refresh :after 'repl/start :after-args [env])))

3. Optional reconfig at relaunch

You can see the above code, in particular line 20, essentially takes care of the third item as well.  It accepts the optional env that you pass in, or otherwise defaults to the :env of the current system, so long as you follow the convention within your myproject.system/start that your resulting system value has a key :env that contains the passed-in config variable.

4. Convenience variables

Now, when starting up the REPL, you’d think we would jump into namespace repl and start hacking from there. However repl.clj contains code that I want to be able to keep in version control, but I don’t want all my hacking code to be in the VCS, so I don’t have to worry about using private keys and such when I’m just doing experimental development. So instead I create a namespace hack that is excluded from the VCS, and add references to the rest of my project there. The namespace declaration looks like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
(ns hack
  (:require
    ; things you'd want in the repl by default
    [ clojure.java.io :as io]
    [ clojure.string :as str]
    [ clojure.pprint :refer [pprint]]
    [ clojure.repl :refer :all]
    [ clojure.test :as test]
    [ repl :refer :all]
 
    ; another .gitignored file with your configs
    [ env]
 
    ; project namespaces that you want to use
    [ myproject.services.database :as db]
    [ myproject.services.facebook :as fb]
    [ myproject.utility.io :as myio]
 
    ; libraries you want to use
    [ clj-time.core :as time]
    [ clj-time.coerce :as timec]
    [ monger.collection :as mc]
    [ monger.operators :refer :all]))

Then all the hacking you do, you put in a comment block so it doesn’t all get executed when you load the file. Notably, at the top of this Clojure block, I add a quick initialization script to create some handy top-level variables after initializing the system, solving issue #4 above. At the end of the day, the block will look like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
(comment
  ; initialization block
  ; just change to e.g. env/prod and re-run to change environs
  (do
    (repl/reload env/dev)
    (def db (-> repl/system :monger :db))
    (def req {:system repl/system}))
 
  ; misc hacking
  (def post (last (mc/find-maps db :posts)))
  (pprint (myproject.controllers.folders/get-folder-json folder req))
  (pprint (db/group-folder-items db folder))
  (pprint (distinct (map #(-> % :image :folder) (mc/find-maps db :uploads)))))

5. Resulting workflow

Now, my Clojure workflow is, launch the REPL, open hack.clj in my editor (I use cursive, but this will work anywhere), and load the flie into the REPL. I decide which env I need to use, and type that into the do block, and execute it. I’ve got my “system”, my DB, etc all connected as top-level variables, and I can start making database calls via the library, or via my own database wrapper code.

By playing around in the hack namespace, I figure out what changes I need to make to the actual project code. I make those changes to the source files, go back to hack.clj pull in the changes by running that top-level do block again and everything is ready for testing it out. If I want to switch to e.g. the production environment to verify everything works there too, then simply change env/dev to env/prod, run the do block again, and you’re on your way.

Clojure, Programming

dfohl

More posts by dfohl

Related Posts

  • Reading line-by-line from a serial port (or other byte-oriented stream)

    By Ben Voigt | 10 comments

    With many .NET developers moving from the traditional (and broken) System.IO.Ports.SerialPort DataReceived event handling to either the correct and more efficient BaseStream.BeginRead / BaseStream.EndRead pair I promoted in my last post or the newer BaseStream.ReadAsyncRead more

  • How to Model NPT Threads in Solidworks

    By rmontifar | 2 comments

    National Pipe Thread Taper or NPT threaded pipes and fittings are deployed in a variety of fields where transportation or containment of liquids, gases, steam, or hydraulic fluid is required. The NPT geometry allows internalRead more

  • Multi-Tiered Linux Backup System – Part I

    By dreynolds | 0 comment

    Backing up important data and memories is an important task that should not be neglected. Just as important as performing Linux backups is verifying that the backups made are good and can be used toRead more

  • Start Zoneminder Recordings with Vera Events

    By dsmoot | 4 comments

    In a previous post I explained how you could configure the security DVR software Zoneminder to trigger recordings from a network connection. While a neat trick, I never really explained why I set this up.Read more

  • nRF51 Development Environment with Eclipse

    By dreynolds | 6 comments

    This blog post aims to describe how to setup a fully functional nRF51 development environment using Eclipse and the GCC toolchain.  We are using a Segger JLink GDB server for the debugging portion, but thereRead more

Leave a Comment

Cancel reply

Your email address will not be published. Required fields are marked *

NextPrevious
  • Home
  • Expertise
  • Blog
  • Careers
  • About Us
Sparx Technologies, LLC. dba Sparx Engineering © 2009 - 2022 | All Rights Reserved
  • Home
  • Expertise
    • Software Engineering
    • Electrical Engineering
    • Chemical Products and Services
    • Biomedical Engineering
    • Mechanical Engineering
    • Production Management
    • Automation
    • Industrial Design
  • Blog
  • Careers
  • About Us
Hardware, Software & Product Development | Sparx Engineering