Continuous integration and code coverage for Emacs packages with Travis and Coveralls

Posted: - Modified: | emacs, elisp

Do you maintain an Emacs package hosted on Github? Would you like to get those confidence-building, bragging-rights-granting, other-developers-inspiring build: passing and coverage: 100% badges into your README file?

It turns out that this is pretty easy with ERT, Cask, Travis CI, undercover.el, and Coveralls.io.

  1. Log on to Travis and enable continuous integration for your repository.
  2. Log on to Coveralls.io and enable coverage testing for your repository.
  3. Set up a git branch, since you'll probably be making lots of small commits while you smooth out the testing workflow.
  4. Define your tests with ERT. See https://github.com/abo-abo/tiny/blob/master/tiny-test.el for an example. For undercover support, you'll want to include something like:
    (when (require 'undercover nil t)
      (undercover "tiny.el"))
    
  5. Define your dependencies with Cask. Include undercover. For example, here's a simple Cask file:
    (source gnu)
    (source melpa)
    
    (development
      (depends-on "undercover"))
    
  6. Add a .travis.yml that specifies how to test your package on Travis. For example, see this .travis.yml and Makefile.
  7. Commit and push.
  8. Check your repository status in Travis to see if it ran properly.
  9. Check your coverage status in Coveralls.io to see if it displayed properly.
  10. Get the badge code from Travis and Coveralls, and add them to your README (probably using Markdown). You can get the badge code from Travis by clicking on your build status badge next to your repository name. Coveralls has prominent instructions for getting your badge. Yay!

Incidentally, if you want to see your test coverage locally, you can (require 'testcover) and then use testcover-this-defun or testcover-start to instrument the macros and functions for coverage. Run your tests, then use testcover-mark-all to look at the results. See the documentation in testcover.el to find out what the coloured overlays mean. Edebug has a test coverage tool too, so you can explore that one if you prefer it.

Additional notes on testing:

Resources:

You can view 3 comments or e-mail me at sacha@sachachua.com.

3 comments

Thanks for pointing me to `testcover`. It's a really nice tool.

Sándor Juhász

2016-02-09T06:55:22Z

Thank you Sacha for sharing this recipe, it helped me a lot to get started with the CI and coverage tools.

I found that Travis is transitioning to a faster and more economic "container based infrastructure" and sudo will not be available in that environment. To install Emacs and Cask to this new platform, evm from rejeep can be used without sudo rights. As an added bonus the builds are much faster and v25 snapshot is supported too.

Links and examples:
https://github.com/rejeep/evm
https://github.com/sviridov...
https://github.com/sandor-j...

Great, thanks for the update! =)