Every time a new build is created it needs to be tested for basic functionality. The point of such testing is to flag serious problems in the new build, and make it easy for the project manager to ensure that the program will pass the first round of testing. If the new build fails initial verification, some companies prefer to withdraw it from the test process and to save QA resources for other tasks. Other companies evaluate the test result that tells testers what problem areas they need to focus on, or what areas to avoid because they are not ready yet.

This first round of testing is time consuming, boring and requires a significant amount of QA resources. Moreover, the test result usually is available only the day after the new build is created.

Automated Build Certification (ABC) is a framework of scripts that automatically installs the new build and runs some test cases to make sure the build is not broken and is ready for the second round of tests. It starts immediately after the new build is created. If builds are done at the end of the day, then the QA team can utilize the overnight time to do some up-front automated testing before accepting the new build for the next test round.

There are significant benefits to having an automated build certification process that does some automated testing before the QA team accepts the build for testing:

· Part of the acceptability testing can be automated and run overnight.  QA no longer has to take a new build and verify that it is acceptable for testing because they can rest assured that if the build happens and the ABC runs successfully, the build is stable and ready for testing.

· Developers are alerted ASAP if there are any problems with the build. Often the development team knows about build problems simultaneously with the QA team and can start working on the discovered problem even before QA reports bugs.

As a result of using ABC, company saves significant time on internal procedures between build creating and starting bug fixing.

The ABC contains configuration file, main script and some testing modules that are responsible for different areas of application testing. The configuration file allows adding and turning on/off testing modules and their parameter configuration. In the configuration file the user can change the order of executing test modules. The file also contains mail lists of dedicated people who receive ABC reports by e-mail.

Each testing module (TM) is an independent script or application that is called by the main script. It can have its own configuration file and creates a separate log file. ABC can use any SilkTest, WinRunner or other automated tool script with some minor updates as testing module.  One of the testing modules is responsible for install/uninstall verification.

The main script drives test modules, analyzes TM log files and sends e-mail notifications with ABC results. It can also create general Web reports for the build under certification. The main script along with the  configuration file allows running testing modules in any order any number of times on any specified machine(s). 

CrystalTest delivers the following ABC components:

1. Main script with hooks to add new modules
2. Configuration file
3. Test module that is able to install/uninstall application under test and verify the install result
4. ABC documentation