“Does your project have a CI system that suffers from an ever-growing set of race conditions? We have the tool for you: it has enabled increased velocity despite project growth. When talking about the GNU HURD kernel, Richard Stallman once said, “it turned out that debugging these asynchronous multithreaded programs was really hard.” With 30+ asynchronous services developed by over 1000 people the OpenStack project is an object lesson of this problem. One of the consequences is race conditions often leak into code with no obvious defect. Just before OpenStack’s most recent stable release we were pushing the boundaries of what was possible with manual tracking of race conditions. To address this problem we have developed an ElasticSearch based toolchain called “elastic-recheck.” This helps us track race conditions so developers can fix them and identify when CI failures are related to the failed patch or are due to a known pre-existing race condition. Automated tracking of over 70 specific race conditions has allowed us to quickly determine which bugs are hurting us the most, allowing us to prioritize debugging efforts. Immediate and automated classification of test failures into genuine and false failures has saved countless hours that would have been wasted digging through the over 350MBs of logs produced by a single test run.”