ETest (or etest, if you like) is the Emacs Testing Framework. It is a small modular system for writing unit tests in Emacs Lisp.
At time of writing ETest consists of two files. etest.el
provides the core test functionality. It defines the function
(actually, macro) etest which is usually one's entrance into a
test run. The file etest-result-mode.el adds functions that
allow the visualisation of a run with syntax highlighting, folding and
comment toggling.
At the time of writing you must download ETest using git:
git clone http://repo.or.cz/r/ETest.git etest
Though it will always be bleeding edge master should always be
in a working state.
To install ETest use one of the following methods:
Once you've fetched ETest change into the new directory and run
make if all goes well run make install (you may need
escalated privileges for this step).
If you don't want to byte-compile for some reason then you can just
copy the .el files into a directory in your load-path.
If you would rather add the unpacked library to your load path then the following will work assuming the etest directory is in your home directory:
(add-to-list 'load-path "~/etest")
If you want to run tests from a lisp buffer add the following to your initialistation file:
;; The only keybinding we make... hopefully
(add-hook 'emacs-lisp-mode-hook
(lambda ()
(local-set-key (kbd "C-c t") 'etest-execute)))
And to have .etest files load emacs-lisp-mode put the
following into your initialistation file:
;; Load lisp mode when editing an etest file
(add-to-list 'auto-mode-alist '("\\.etest$" . emacs-lisp-mode))
First you must evaluate (require 'etest) then you can start
using the etest macro.
Using ETest is very simple. Once you have installed the modules then you can simply run your first test like this:
(etest (ok 1))
This should pop up a results buffer showing you the outcome of the
run. In this case all should be ok because, well, 1 is a pass
according to the ok test.
Generally (at least the way I work) is to have a .etest file
which matches all but the extension of the filename of the file I am
to test. So if I were testing a file called find-cmd.el I would
write my tests in a file called find-cmd.etest (in the same
directory as find-cmd.el) and then when I hit C-c t (in
emacs-lisp-mode) the run would happen and the results pop up in
their normal fashion.
Thanks to etest-execute.el there are several options for the location of an etest file (one at a time). In order of execution they are:
1. The buffer local variable etest-file is set to the filename
of the etest file.
2. A matching etest file is in the directories listed in
etest-load-path.
3. A matching etest file is in the current working directory
(default-directory).
4. A matching etest file is in one of the directories mentioned in
load-path.
At the moment it's impossible to have two etest structures defined in
one file and see the results in one results buffer (at least in
the result mode that is bundled with ETest) as an erase-buffer
is executed (effectively) for each invocation of the etest
macro.
The results buffer is where you can see (and manipulate) the results of a test run in a human friendly format. It will always popup when etest is run and let you know how things went.
Given the hypothetical tests:
(etest
("Numeric comparisons"
("Integers"
(ok (> 1 0) "one is more than 0")
(ok (< 1 0) "one is less than 0"))
("Floats"
(ok (> 10 9.99))
(ok (< 1 -5.2) "one is less than 5.2"))))
Once run give us the following in the results buffer:
* Numeric comparisons
** Integers
ok ................ one is more than 0
# got: 't'
not ok ............ one is less than 0
# got: 'nil'
** Floats
ok ................ (ok (> 10 9.99))
# got: 't'
not ok ............ one is less than 5.2
# got: 'nil'
All headings are foldable as are comments.
etest-rm-comment-visibility-types and use
the car of that list to determine the visibility of comments.
Tests are always run within the etest form and usually always
evaluate their arguments. Tests will always have a defined number of
required arguments (for example ok requires one argument. Each
test also allows for one optional argument which is a custom
documentation string. If this argument is omitted then ETest will
generate one in its place. So, for example, if you used
(etest (ok 1)) the doc string would be "(ok 1)" if you used
(etest (ok 1 "Foo")) the doc string would be "Foo".
Tests can be grouped within headings by simply using a string as the first element of a form like this:
(etest
("Simple tests"
(ok 1)))
You can nest headings to your hearts content.
This will produce a set of results that follow the same hierarchical pattern as the tests themselves. For example the above produces the following structure:
(("Simple tests"
(:result t :comments "got: '1'" :doc "(ok 1)")))
The output in the results buffer is:
* Simple tests
ok ................. (ok 1)
# got: '1'
These basic tests allow you, basically, to check if a value is either non-nil or nil.
ok will only pass if its argument produces a non-nil result.
(etest (ok (+ 1 1)))
null will only pass if its argument produces a nil result.
(etest (null nil))
The following functions map to their lisp counterparts and so don't really require much explanation. Evaluate each of the examples to watch them pass.
Each take two forms which, post evaluation, are the objects to compare.
(etest (eq 1 1))
(etest (eql 1.1 1.1))
(etest (equal '(1 2) '(1 2)))
These two functions each take one form.
This test will pass if an exception is raised. For example, cause a
divide by zero error ((arith-error)):
(etest (error (/ 1 0)))
This test will pass if no exception is raised. For example, a valid division will not raise an error:
(etest (noerror (/ 0 1)))
like takes two arguments a string and a regexp to test it against:
(etest (like "Hello" "^H\\(e\\)"))
Produces this in the results buffer:
ok .................. (like "Hello" "^H\\(e\\)")
# searching: 'Hello'
# match 1: 'e'
The grouping within the regular expression only affects the comments.
Defining your own tests is fairly trivial and where ETest becomes really useful.
Each test must return a plist that has :result and, optionally,
:comments in it.
:result represents whether the test passed or failed non-nil
for a pass and nil for a fail.
The optional :comments are newline separated strings that might
help the user in their diagnosis of a problem. Comments should follow
the conventions set by the 'builtin' tests using keywords such as
'got:'.
To have ETest recognise the test as valid the deftest function
should be used. For example if I wanted to create a function that
took two arguments and tested the first was numerically greater than
the other I might do this:
(defun etest-greater-than (one two)
(let* ((res (> one two)))
(list :result res
:comments (unless res
(format "one: '%d'\ntwo: '%d'" one two)))))
We let emacs take care of type errors (and any other type of error) which is what all of the builtins do. Also it's worth noting that if you want your arguments evaling you have to do it yourself.
Now we let ETest know this new function exists:
(deftest '(> 2) 'etest-greater-than)
So, the new function will be called >, it will take two arguments
and it calls maps to etest-greater-than.
(deftest '(> 2) 'etest-greater-than)
Now you can mix > with other tests:
(etest
(ok "something")
(> 1 2 "one is more than two"))
Which in the results buffer produces:
ok .................. (ok "something")
# got: '"something"'
not ok .............. one is more than two
# one: '1'
# two: '2'
If you wrap a test in a todo keyword:
(todo (ok nil))
Then the test will appear to pass but with an extra keyword in the
result (:todo) with the value t.
The result mode takes account of this by highlighting todoed tests and showing you what happened when ETest actually ran the test (catching all exceptions).