The task is to measure changes in of a system during daytime execution.

Test system description

Tests are performed on java desktop application which is started with cmd command:

java -cp [AUT dependencies, .jars] [main app class]

Interaction with application GUI is done with library so test is started like:

java -cp [AUT &  dependencies/jars] org..TestNG .xml

And application is launched in the code of tests:

new ClassReference("[main app class]").startApplication();

So test project is a part of application and the question is how much test project performance of AUT?

Test execution

Tests are executed every 2 minutes within a single run of testng command. In testng.xml file <test> blocks a repeated so that execution would not restart test (as well as AUT). But after ~40 minutes of execution testing fails because of wait timeout expiration. Here is a processor & RAM load graph:

enter image description here

And heap usage dynamics:

------------------------------------------------------------------------------------
|       ||     Exec time [R# - Run number] [S/F/C - at run start/finish/crash]     |
------------------------------------------------------------------------------------
| [MB]  || R#2 S | R#2 F || R#10 S | R#10 F || R#21 S | R#21 F || R#22 S | R#22 C ||
------------------------------------------------------------------------------------
| Max   ||  455  |  455  ||   455  |   455  ||   455  |   455  ||   455  |   455  ||
------------------------------------------------------------------------------------
| Total ||  286  |  374  ||   373  |   374  ||   454  |   454  ||   454  |   454  ||
------------------------------------------------------------------------------------
| Free  ||  123  |  240  ||    86  |    94  ||    67  |    51  ||    70  |    50  ||
------------------------------------------------------------------------------------
R#2 - 04:00
R#10 - 20:00
R#21 - 42:00
R#22 - 44:00

Update:

Started testing with minimal number of actions in AUT (just clicking menu buttons). After an hour of execution heap usage was:

  • maximum – 455
  • total ~430
  • free ~200

After 02h:00m:00-02h:20m of execution heap usage was:

  • maximum – 455
  • total – 455
  • free ~35-70

After 02h:32m test failed:

  • maximum – 455
  • total – 455
  • free – 53



Source link https://sqa.stackexchange.com/questions/20447/how-does-testng-influence-on-project-performance

LEAVE A REPLY

Please enter your comment!
Please enter your name here