Previous: Configuration files, Up: The ARTE Core

4.6 Execution modes

As said before, ARTE allows three different execution modes (a.k.a ‘runtypes’), depending on the needs for each module to be analized, or depending on the type of testing to be done. The different runtypes allow to configure how the external module will be started before running the tests, and stopped after having run them (using the provided [module].start and [module].stop commands in the suite-specific configuration file).

‘manual’
In this mode, ARTE will neither start nor stop the external module. The external module to be tested must be already started before calling ARTE with this runtype. This runtype is specially useful when debugging the external module.
‘always’
In this mode, ARTE will start the external tested module before each test, and stop it after each test. In other words, each test is executed in a completely independent execution of the external module.
‘once’
In this mode, ARTE will start the external module before the first executed test and will stop it after the last one. This runtype is specially useful when performing load-tests or checking the behavior of the external module when multiple tests are executed over the same single execution1.

Footnotes

[1] When using ARTE with the first external module in Panda Security S.L., it happened sometimes that running all the tests with ‘once’ runtype gave different outputs than running the all the tests with ‘always’ runtype, due to inter-test problems. For example, running ARTE in ‘once’ runtype, first test was causing a problem in the external module which was also causing some of the next tests executed to fail, while when executed in ‘always’ runtype, the external module was a completely new instance of execution with no corrupted internal state, so all the tests passed correctly.