Ayler - A simple Namespace Browser

Notes from my (almost) first clojure project.

About Me

  • Operations consultant.
  • Doing some development on my (very limited) spare time.

Ayler

http://github.com/babysnakes/ayler

What is Ayler?

  • Namespace browser for your project.
  • Connects to your project via nrepl.
  • Requires minimal (or no) dependencies in your project.
  • Displays all the loaded namespaces.
  • Displays all public members of a selected namespace.
  • Displays docstring and source of selected var.
  • Allows you to search and load any namespace from a list of all namespaces in your classpath (requires 2 dependencies).

~

Demo

Agenda

  • Inspiration
  • Motivation
  • Workflow
  • Error handling
  • State
  • Testing
  • Javascript
  • Last thoughts

~

This is a conversation!

Inspiration

labrepl

  • A set of tutorials by Relevance
  • One of the tutorials is a namespace browser
  • In process
  • Very basic set of features
  • Nice web interface

~

cljs-ns-browser

  • In process
  • Many dependencies (version clashes with project dependencies)
  • Desktop application (swing)
  • Many features

~

Motivation

  • Avoid always browsing github project files or api docs to see what is available in a library
  • A nice project to learn clojure and (later) clojurescript
  • Web interface
  • No (or little) dependencies
  • External to your project

~

Ayler uses

  • Clojure web stack:
    • Ring
    • Compjure
    • Ring-json
    • http-kit
  • Nrepl (connects to project)
  • AngularJS single page application
  • CLI Utilities

~

Workflow

  • Reload on changes
  • Running test after changes

~

Workflow (old)

  • lein-ring
  • Reload routes on every request:
    
    (defmacro var-route
      [route]
      (if (development?)
        `(var ~route)
        route))
                    
  • Load every change to the nrepl (C-c C-l)
  • Run tests in repl - This may cause invalid test results (e.g. a function may be deleted but it's still in the repl namespace).
  • Once in a while (or when something seems suspicious) run lein test from a shell.

~

Stuart Sierra's workflow

As described in the famous blog post.

The System Constructor

The only global variable in the application. Should hold all the data in the application.


(defn system
  "Returns a new instance of the application."
  []
  {:settings {:port 5000}})
            

(defn init
  "Construct development env."
  []
  (alter-var-root #'system
    (constantly (-> (app/system)
                    (assoc :remote [6001 "localhost"])))))
            

Start/Stop the system


(defn start
  "Start all components of the application. Returns the updates system."
  [system]
  (when-let [level (:log-level system)]
    (timbre/set-level! level))
  (when-let [remote (:remote system)]
    (apply client/set-remote remote))
  (let [server (run-server app (:settings system))]
    (timbre/info (str "Ayler started on port " (get-in system [:settings :port])))
    (assoc system :stop-server-fn server)))
            

(defn stop
  "Stops all components of the application. Returns the updated system."
  [system]
  (if-let [stop-server-fn (:stop-server-fn system)]
    (stop-server-fn))
  (assoc system :remote (client/extract-remote)))
            

Dev Profile


:profiles {:dev {:source-paths ["src/dev"]
                 :dependencies [[ring-mock "0.1.5"]
                                [org.clojure/tools.namespace "0.2.4"]
                                [org.clojure/java.classpath "0.2.1"]]}
            

(ns user
  (:require [ayler.app :as app]
            ...
            [clojure.tools.namespace.find :refer (find-namespaces-in-dir)]))

(def system nil)

(defn init
  ...)

(defn start
  ...)

(defn stop
  ...)

(defn go
  "Initialize and run"
  []
  (init)
  (start))

(defn reset []
  (stop)
  (refresh :after 'user/go))

(defn run-all-tests
  ...)
            

Workflow

  • Resetting the environment after every change
  • Running tests only in repl (no need for lein test)
  • Sometimes full VM reset is required

~

Gotchas

  • Not setting the system correctly
  • AOT
  • Problems in user.clj prevents the repl from loading.
  • Check the blog and the README in the tools.namespace repository for problems and possible solutions.

~

Error handling

Hardly ever use exceptions

Indicate errors with data structures


(defn eval-on-remote-nrepl
  "Evaluates the op and code on the remote nrepl.
  ..."
  [op code]
  (if (empty? @_remote)
    {:status :not-connected}
    (try
      #_(...)
      (with-open [conn (apply repl/connect @_remote)]
        #_(...))
      (catch java.net.ConnectException e
        (do #_(...)
            {:status :disconnected})))))
            

Other Error Handling Concepts

~

State

The Caves of Clojure

Ayler (things are simpler)

State is usually in the database or session

Every layer is in charge of it's own state

Nearly all of the functions are pure

In every layer there's a few messy functions that collect all the required state.

~

nrepl-client.clj

(defonce ^:private _remote (atom []))

;; ... getter and setter
            
api.clj

(defn- var-doc
  "Returns the docstring of a fully qualified var name"
  [namespace var]
  (->  (construct-varname namespace var)
       (queries/query-docstring)))
            

(defroutes routes
  #_(...)
  (GET "/api/doc/:namespace/:var" [namespace var]
       (response (var-doc namespace var)))
  #_(...))
            

Stuart Sierra - Clojure in the Large

Announced today - Component - Framework for managing lifecycle of stateful objects.

Testing

Test only the pure methods

Rely on the fact that the non-pure methods are really simple

About 0.5 to 1 test/code ratio

Compared to 1 to 1 javascript test/code ratio

Integration tests

~

JavaScript

AngularJS

JSON

Ring-json (There are more fully featured solutions - liberator)


(defroutes routes
  #_(...)
  (POST "/api/disconnect/" _ (response (client/disconnect))))

(def app
  (-> routes
      wrap-json-response
      wrap-json-params))
            

Protection against CSRF with customized ring-anti-forgery (submitted PR)

~

JSON 2


{
    "status": "done",
    "response": "([x])\n  Returns a number one greater than num. Does not auto-promote\n  longs, will throw on overflow. See also: inc'"
}
            

{
    "status": "error",
    "response": "IllegalArgumentException No such namespace: clojure.coree  clojure.lang.Var.find (Var.java:153)\n"
}
            

~

JSON 3


apiClient.handleResponse = function(response, handler) {
  switch(response.status) {
  case "disconnected":
    $rootScope.$broadcast("connect", {disconnected: true});
  case "not-connected":
    $rootScope.$broadcast("connect");
  case "done":
    handler(response.response);
    break;
  case "error":
    apiClient.handleError(response.response);
    break;
  default:
    alert("Unknown response: " + response);
    break;
  };
};
            

Last Thoughts

OO vs Functional

Project Layout

THANK YOU

Questions?