bw logo

Chapter 6. OpenAutomate

OpenAutomate is a testing and benchmarking automation library produced by NVIDIA. OpenAutomate is shipped as part of the BigWorld Technology license. See http://developer.nvidia.com/object/openautomate.html for more details.

BigWorld is using and extending the Open Automate API to test the BigWorld client in house. The res\scripts\client\OpenAutomate.py script contains the code which runs the tests based on the Open Automate instructions.

There are several ways to use the BigWorld Technology Client with the OpenAutomate framework to test a game:

6.1. Using oaman to execute tests.

NVIDIA provides a tool called oaman which allows setting specific game settings and running tests. The oaman tool is available in src\lib\third_party\openautomate\oaman\oaman.exe.

Using oaman can be done as follows:

  • Start oaman.exe

  • Choose ApplicationAdd OA Application

  • Choose the BigWorld client executable being used to run your game.

6.2. Scheduling command line tests.

Running nightly testing to detect performance degregations and stability issues is a recommended practice for game development. The BigWorld Technology Client can read and execute open automate commands (and additional commands) based on xml files provided as part of a command line. Following is the syntax for running a test using an xml instructions file:

<BigWorld Technology Client> -openautomate INTERNAL_TEST <text xml file>

For example:

bigworld.exe -openautomate INTERNAL_TEST \
	<fd res>\scripts\testing\openautomate\simpleSetAutoDetect.xml

The above example illustrates how to auto detect the client settings and use the highest video mode. Please refer to the corresponding

currentCommand == "CMD_AUTO_DETECT" 

in OpenAutomate.py to understand the implementation. This command should be used before running other tests to make sure that we are using the same graphics and video settings when running tests.

Running the command:

bigworld.exe -openautomate INTERNAL_TEST \
	<fd res>\scripts\testing\openautomate\testAdvancedRunBenchmarks.xml

will run multiple benchmarks and quit.

Customers wishing to add more tests can add logic to the OpenAutomate.py file. The function testOAGetNextCommand() available at src\lib\open_automate\open_automate_tester.cpp contains the implementation which reads the xml files and sends commands to the OpenAutomate.py file. Changing this function might be required for advanced customers wishing to add additional testing capabilities to their xml files.