Introduction
ARTE is a unit/system testing framework designed to test software modules in GNU/Linux systems. It's based on the automatic execution of test suites/collections and helps the improvement of Your software with fast detection of regression problems.
ARTE was originally developed at Panda Security, which kindly published the source code under the GPLv3+ license. Therefore, ARTE is free software, and can be extended by anyone as long as the rules of the GPLv3+ license are satisfied.
ARTE uses the Check unit-testing framework as the core test executer, and TCL (Tool Command Language) as the test interpreter.
Black-box system testing
The standard unit-testing frameworks allow the definition of unit tests which are usually integrated in such a way that API functions of the module are called, and deciding if the test is passed or failed depending on the returned value/data of the function. Those systems are useful to test libraries with a well-defined API, but they become a problem when trying to test bigger software modules in a black-box manner, with other kinds of interfaces.
Where ARTE succeeds is where others have problems. ARTE is designed to work with black-box software modules, and becomes fully powerfull when testing client/server architectures:
- It may be configured to test a SIP server, sending SIP requests to the server and checking the returned SIP response.
- It may be configured to test a Web server, where malformed URIs are passed, and check if the HTTP return code sent by the server is the expected one.
- It may be configured to test a computer Virus search engine, injecting infected files to the engine and checking the result of the detection.
- It may be configured to test a document parsing utility, passing different document files in different tests.
In other words, ARTE is not a usual unit testing framework, but something in between unit tests and system tests.
Features in a nutshell
Automatic execution: You can configure your build system to run ARTE after every compilation, or just before generating a new release of the software. ARTE will run all the available tests and generate a XML/HTML report with the results.
RTA: RTAs or Regression Test Archives are the single tests run by the ARTE framework. These are password-zipped files which contain the TCL script to run, test description XML file, test configuration and additional needed files for the tests. The password protects the contents of the file, so that any kind of content can be stored inside (even virus-infected files).
New tests: Adding new tests in the ARTE framework is as easy as generating a correct RTA file, and placing it in the configured test archive folder. When ARTE runs, it will look for all the RTA files and execute them.
Test scripts: Tests are just TCL scripts where specific commands are available. This allows You to use a strong scripting language in Your tests, with lots of capabilities.
Extending the scripting language: If the built-in TCL commands are not enough, You can extend the scripting language with your own commands, by developing C-language plugins for ARTE. This built-in plugin infrastructure is really easy to use with a well defined plugin API.
Execution modes: You can configure ARTE to automatically start the module to analize before any test, or to start/stop it for each one of the tests. This two main execution modes allow You to avoid inter-test dependencies (for example, when one of the tests made the tested module crash, or left it in an inconsistent state).
