In similar scenarios I’m in exactly the same freakin scenario right now. Sigh. A key for me is to convince the business of what needs to be tested where, oterwise… they’ll end up directing testing everything throug the … So the two key main points I am making to my business are the need for automation that performs well in terms of two key factors:

Success in both speed and reliability for software test automation is hard and most technology companies struggle with both.

The overall goal of software test automation is to provide faster feedback to the business at all levels from the development team to the various levels of management, appropriately summarized at each level.
For software test automation itself, using the UI to create test data or set application state is incredibly slow and in some cases not even possible and will thus not provide the timely feedback that is required.
This is frequently a 10x+ (and often 100x+) factor in terms of testing speed and dramatically affects the ability to both develop and test automation. If automation takes 20 minutes to reach a specific spot in a workflow instead of 7 seconds this will make quality automation development impractical. Consider also then trying to run that same test in 20 different device-browsers when that means 20×7 = 2+ hours for that 7 second test. Imagine wanting to try 10 different changes to get it right (common).
We do not control or select our customers device, browser, version or other client-side characteristics such as different javascript and rendering engines. Customers devices are now updated daily in the background, communication is asynchronous and network connectivity may vary. This greatly limits the ability to have reliable UI and ultimately prevent us from having 100% reliability despite out best efforts. This is not a problem that any one company created and it is not a problem that one company can solve – through we can allow and account for it. One of the main implications is to have as few UI as possible in favor of and Integrated that run in seconds instead of hours and days (test pyramid vs ice-cream cone). The few automated UI we have should be carefully selected to ensure they avoid duplication and test data combinations through the front end.

Internally, for test automation development, it is essential to observe and structure testing based on the test pyramidenter image description here

and the agile testing quadrants

enter image description here

to ensure getting the benefits of swift feedback throughout the the development process.

There are two key technical problems to solve:

  • setting up test conditions and state without using the UI
  • controlling test and application state through APIs

Traditionally the need for test automation has not be enough of a critical business success factor to justify actually spending serious application development effort, time and money on addressing these two key technical issues (they are certainly solveable with enough effort and determination). That has now changed and test automation is increasingly recognized at a hugely critical busines success factor underpinning so many other efforts and plans of todays businesses.

For the detailed strategy of dividing up unit, integrated tests and UI tests, a few thoughts come to mind?

  • Create some examples of functionality with (e.g.) 60 unit, 12 integrated and 4 UI tests to show the test pyramid in action
  • Make sure unit tests mock and stub dependencies
  • Ensure that appplication and automation engineers share code reviews of each others work
  • Expose application environment stability for functional tests constantly in the application/automation team space area
  • Expose test status information constantly in the application and automation development area
  • Ensure that application developers write units tests and use code coverage and quality measurement tools as a required part of their continuous integration code development process.
  • Ensure that application developers and UI automation developers are physically close to each other for easy drop-in chats.
  • Consider creating a formal triage system (e.g. during backlog refinement) to determine what to test where (in terms of unit, int and uat tests) at least until you can do it instinctively.

Source link


Please enter your comment!
Please enter your name here