I would be very handy to have something like make test TESTS=rerun_fails That would just run the failed tests. We can also think of another for defining which file should be read for the test report.
I've been working on this for quite some time now, by trying to get Samba to provide everything that is required by the "testr" tool that is part of the subunit ecosystem - see https://launchpad.net/testrepository
This is mostly implemented on a framework level. The remaining thing to do is to fix all testsuites to allow the running of individual tests.
This is now implemented at the framework level. You can use "selftest.pl --load-list=FILE" where FILE contains a list of test ids to run.
Note that this doesn't quite work yet in practice because some individual tests don't support the load list functionality.