Continuous integration and code coverage for Emacs packages with Travis and Coveralls

Do you maintain an Emacs package hosted on Github? Would you like to get those confidence-building, bragging-rights-granting, other-developers-inspiring build: passing and coverage: 100% badges into your README file?

It turns out that this is pretty easy with ERT, Cask, Travis CI, undercover.el, and Coveralls.io.

  1. Log on to Travis and enable continuous integration for your repository.
  2. Log on to Coveralls.io and enable coverage testing for your repository.
  3. Set up a git branch, since you’ll probably be making lots of small commits while you smooth out the testing workflow.
  4. Define your tests with ERT. See https://github.com/abo-abo/tiny/blob/master/tiny-test.el for an example. For undercover support, you’ll want to include something like:
    (when (require 'undercover nil t)
      (undercover "tiny.el"))
    
  5. Define your dependencies with Cask. Include undercover. For example, here’s a simple Cask file:
    (source gnu)
    (source melpa)
    
    (development
      (depends-on "undercover"))
    
  6. Add a .travis.yml that specifies how to test your package on Travis. For example, see this .travis.yml and Makefile.
  7. Commit and push.
  8. Check your repository status in Travis to see if it ran properly.
  9. Check your coverage status in Coveralls.io to see if it displayed properly.
  10. Get the badge code from Travis and Coveralls, and add them to your README (probably using Markdown). You can get the badge code from Travis by clicking on your build status badge next to your repository name. Coveralls has prominent instructions for getting your badge. Yay!

Incidentally, if you want to see your test coverage locally, you can (require 'testcover) and then use testcover-this-defun or testcover-start to instrument the macros and functions for coverage. Run your tests, then use testcover-mark-all to look at the results. See the documentation in testcover.el to find out what the coloured overlays mean. Edebug has a test coverage tool too, so you can explore that one if you prefer it.

Additional notes on testing:

2015-02-03 Better Emacs Testing -- index card #testing #emacs

2015-02-03 Better Emacs Testing – index card #testing #emacs

2015-02-04 Yay, testing in Emacs -- index card #testing #emacs

2015-02-04 Yay, testing in Emacs – index card #testing #emacs

Resources: