[dev-context] regression testing Makefile

Sanjoy Mahajan sanjoy at mrao.cam.ac.uk
Sun Feb 4 17:43:22 CET 2007


The contexttest repository now contains a Makefile to use for checking
one context tree against another.  There's a README alongside it but
here's the quickstart

  svn export . /tmp/testdir
  cd /tmp/testdir
  make -k A=20070112 B=20070123 >& world.log

(the -k is so it doesn't stop on the first error)

It will

1. (if not already done) Download ConTeXt versions 2007.01.12 and
   2007.01.23 from the museum, put each in its own subtree, make formats
   for each

2. Do the following for each test-NNN.tex file:

   a. Run it through 2007.01.12 to make test-NNN-20070112.pdf
   b. Run it through 2007.01.23 to make test-NNN-20070123.pdf
   c. Feed those pdfs to utils/compare-pdfs.sh to generate a bitmap diff
   d. Report differences found by compare-pdfs.sh (dB on each page)

It was interesting running it.  I haven't sorted through all the output
yet, but a one point was that a few unicode fonts are a problem (maybe
because I don't have them, but it makes texexec loop for a long time
doing mktextfm and the like).  More details to follow.

Notes: 

If you want to view the image diffs as they are generated, set the
viewer shell variable in compare-pdfs.sh (I often use xloadimage).

I'd be very surprised if the Makefile worked on anything but Linux (uses
features of bash + GNU make).

I was pretty careful about re-entrancy, so the following works fine at
least in my testing:

   make -k -j 4 A=20070112 B=20070123

-Sanjoy

`A nation of slaves is always prepared to applaud the clemency of their
 master who, in the abuse of absolute power, does not proceed to the
 last extremes of injustice and oppression.'  (Gibbon)  


More information about the dev-context mailing list