The patches in bug 887054 (or as I like to call it, parxpc) have just landed!
The good news is that you can now run all these tests in parallel, and I found that if you have pretty fast storage (like an SSD), the gained speedup is at least approximately equal to the number of (actual, not hyperthreaded) cores in your computer.
For example, I went from this:
./mach xpcshell-test --sequential 305.04s user 78.60s system 49% cpu 12:50.18 total
./mach xpcshell-test 359.51s user 76.25s system 246% cpu 2:56.76 total
Even better, I've seen an almost two time speedup on the Ubuntu test slaves that we have, which are single core.
You might be wondering what allowed us to get this enormous speedup, and the answer (from what I found so far) is basically waiting on IO. Most of the XPCShell based tests do tasks like:
- working with the filesystem
- handling extensions and plugins
- parsing static files
- sending network requests
Really few do CPU intensive tasks, like encoding and decoding some information.
You might be wondering what you have to do to benefit of this. It's easy! Just update your working tree to the latest version of mozilla-central and the next time you run the XPCShell tests with mach, they will run in parallel.
This change should not introduce any new test failures, so if you encounter any, make sure it's not one of the known (filed) intermittents and please file a bug about it. Don't forget to cc me (:mihneadb).
If you have trouble, there's the handy --sequential flag that you can pass to mach in order to run tests one by one.
This is not yet enabled in automation because of some intermittent failures that only happen some test slaves. I will get back with a post with more details and some info on the whole process of parallelizing the harness.
Feel free to post a comment with the speedup you got (--sequential makes it easy to benchmark), I'm really curious!