cort.el is simplify unit test framework and works with Emacs-22 or above.

cort.el is one-file simply test package, there’re so many merits.

• It works with Emacs-22 or above.
• Not need to download big package, many files before testing. Only 1 file.
• You can test your package as close to vanilla’s Emacs.
• If it fails or error tests, it would output the simple and intuitive error message.

(ert.el is attached as standard to Emacs, operates from Emacs-24 and I think that it is difficult to understand the displayed error.)

1. Put cort.el your package root folder.
2. Create Makefile, .travis.yml if you need.
3. Create test cases definition file as [package-name]-tests.el.

(Please look at the file of this repository for a practical example.)

Makefile sample is shown below.

TOP       := $(dir $(lastword $(MAKEFILE_LIST)))

EMACS     ?= emacs

LOAD_PATH := -L $(TOP)
BATCH     := $(EMACS) -Q --batch $(LOAD_PATH)

ELS       := cort.el         # compiling .el list
ELCS      := $(ELS:.el=.elc)

all: build

build: $(ELCS)

%.elc: %.el
    @printf "Compiling $<\n"
    @$(BATCH) -f batch-byte-compile $<

check: build
# If byte compile for specific emacs,
# set EMACS such as `EMACS=26.1 make check`.
    $(BATCH) -l cort-tests.el -f cort-run-tests

clean:
    -find . -type f -name "*.elc" | xargs rm

.travis.yml sumple is shown below.

language: generic
sudo: false

env:
  global:
    - CURL="curl -fsSkL --retry 9 --retry-delay 9"
  matrix:
  - EMACS_VERSION=23.4
  - EMACS_VERSION=24.5
  - EMACS_VERSION=25.3
  - EMACS_VERSION=26.1
  - EMACS_VERSION=master
install:
  - $CURL -O https://github.com/npostavs/emacs-travis/releases/download/bins/emacs-bin-${EMACS_VERSION}.tar.gz
  - tar xf emacs-bin-${EMACS_VERSION}.tar.gz -C /
  - export EMACS=/tmp/emacs/bin/emacs

script:
  - make
  - make check

cort-test.el sumple is shown below.

;; require depends package
(require 'cort)

;; if you need temporary functions for a test, define this.
(defun quote-a ()
  'a)

(defmacro sym (x)
  `',x)

;; define test cases.
(cort-deftest simple:equal
  (:equal '(a b c) '(a b c)))

(cort-deftest simple:=
  (:= 100 100))

(cort-deftest quote-a:0
  (:eq 'a 'a))

(cort-deftest quote-a:1
  (:eq (quote-a) 'a))

(cort-deftest sym:1
  (:eq (sym a) 'a))

(cort-deftest sym:4
  (:equal (sym (a b c)) '(a b c)))

(cort-deftest error-test
  (:= (+ 1 2) 5))

(cort-deftest err:1
  (:cort-error 'void-function
          (a 'a)))

(cort-deftest err:3
  (:cort-error 'arith-error
          (/ 1 0)))

(cort-deftest cort-if:2
  (:eq 'a
       ('b
        :cort-if (nil 'c)
        :cort-if (t 'a))))

(cort-deftest cort-emacs=:0
  (:= 10
      (0
       :cort-emacs> (0 10))))
;; ...

cort-deftest will receive test-name and test-configuration, and add-to-list to cort-test-cases defined at inside of cort.el.

Therefore, cort-deftest same test case, not running test twice. test-name shouldn’t be a unique name.

test-configuration accept the list of the form (:KEY GIVEN EXPECT), expect to return t when eval (KEY GIVEN EXPECT).

By defining like this, any comparison function can use that returns a boolean value such as eq, equal, or =.

This flexible test notation is one of the important merits of cort.el.

If you pass a list of the form (:cort-error 'ERROR-TYPE FORM) to cort-deftest, 'ERROR-TYPE accepts symbol such as error symbol and expects 'ERROR-TYPE error to occur when evaluating (FORM).

If you want to change the expected form according to the variable (or function returns boolean value), use the :cort-if statement.

(cort-deftest cort-if:1
  (:eq 'a
       ('b
        :cort-if (t 'a))))
;; compare with `eq' 'a and 'a
;; 'a is adopted because first cort-if's VAR is t

(cort-deftest cort-if:2
  (:eq 'a
       ('b
        :cort-if (nil 'c)
        :cort-if (t 'a))))
;; compare with `eq' 'a and 'a
;; 'a is adopted because second cort-if's VAR is t
;; first cort-if statement is ignored

(cort-deftest cort-if:3
  (:eq 'a
       ('a
        :cort-if (nil 'c)
        :cort-if (nil 'd))))
;; compare with `eq' 'a and 'a
;; any cort-if statement is ignored because any cort-if's VAR is nil.
;; so 'a is adopted, default value.

(cort-deftest cort-if:4
  (:eq 'a
       ('b
        :cort-if (t 'a)
        :cort-if (t 'b))))
;; compare with `eq' 'a and 'a
;; 'a is adopted because first cort-if's VAR is t
;; second cort-if statemment is ignored, because first cort-if's VAR is t.

You can specify many :cort-if statement, and you should specify a list like (COND FORM) for each. When the first element of the list is t, it is adopted as the form expected by the second element of it.

If all the first elements are nil, the default value is adopted.

(You can use :cort-if statement for GIVEN or both GIVEN and EXPECT. However, such test cases are confusing you in many cases, so you should not use them.)

If you want to change the expected by Emacs version, use the :cort-emacs* statement. The following symbols are provided.

• :cort-emacs<
• :cort-emacs<=
• :cort-emacs=
• :cort-emacs>=
• :cort-emacs>

(cort-deftest cort-emacs:a0
  (:= 10
      (0
       :cort-emacs> (0 10))))

(cort-deftest cort-emacs:a1
  (:= 10
      (0
       :cort-if ((not
                 (funcall (intern "version<") emacs-version "0"))
                10))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(cort-deftest cort-emacs:b0
  (:= 10
      (0
       :cort-emacs<= (0 10))))

(cort-deftest cort-emacs:b1
  (:= 10
      (0
       :cort-if (((funcall (intern "version<=") emacs-version "0")
                 10)))))

cort-emacs:a0 will be converted to cort-emacs:a1. Likewise, cort-emacs:b0 is converted to cort-emacs:b1.

So you can write :cort-if and :cort-emacs* statement mixed and the earliest value in the list is adopted for expected value.

Please refer to version-to-list in subr.el (Emacs source) to see the value that :cort-emacs* can receive. For example, values like 26.1, 1.0pre2, 22.8beta2 are interpreted correctly. (however, a value not including space)

When writing many test cases, it is troublesome to write common parts many times.

Therefore, you can let the macro make the test case as shown below.

(cort-deftest leaf-test/:if-1
  (:equal
   (macroexpand-1 '(leaf foo :if t))
   '(if t
        (progn
          (require (quote foo) nil nil)))))

(cort-deftest leaf-test/:if-2
  (:equal
   (macroexpand-1 '(leaf foo :if (and t t)))
   '(if (and t t)
        (progn
          (require (quote foo) nil nil)))))

(cort-deftest leaf-test/:if-3
  (:equal
   (macroexpand-1 '(leaf foo :if nil))
   '(if nil
        (progn
          (require (quote foo) nil nil)))))

;; ...

;; Almost test case is (cort-deftest NAME (:equal (macroexpand 'FORM) 'EXPECT))
;; -> Create macro to (FORM 'EXPECT) convert to (:equal (macroexpand 'FORM) 'EXPECT)

;; test target macro
(defmacro package-require (package)
  `(require ,package))

;; Macro to expand FORM and compare it with EXPECT for equal test case
(defmacro match-expansion (form expect)
  `(:equal (macroexpand ',form) ,expect))

(cort-deftest match-expansion0
  (match-expansion
   (package-require 'use-package)
   '(require 'use-package)))

(cort-deftest match-expansion1
  (:equal (macroexpand '(package-require 'use-package))
          '(require 'use-package)))

match-expansion0 and match-expansion1 are equivalent since macros are expanded.

(You can also use a function that returns a list to be accepted by cort-deftest see cort-test.el.

However, test definitions and test runs should usually be separated, and you should not run all forms to immediate when you define a test.

Therefore, we usually recommend using macros.)

All srt suffix flag is renamed to cort suffix.

:error flag has changed to :srt-error so please fix testcase.

;; srt v1.0 notation
(srt-deftest err:1
  (:error 'void-function
          (a 'a)))

;; srt v2.0 notation
(srt-deftest err:1
  (:srt-error 'void-function
              (a 'a)))

Bundling Emacs-22.1 on macOS 10.13 (High Sierra), we support this.

There’re Japanese readme(Readme-ja.org)(obsolete).

We welcome PR! travis CI test cort-test.el with all Emacs version 22 or above.

I think that it is difficult to prepare the environment locally, so I think that it is good to throw PR and test Travis for the time being! Feel free throw PR!

Advice and comments given by Emacs-JP’s forum member has been a great help in developing cort.el.

Thank you very much!!