In similar scenarios I’m in exactly the same freakin scenario right now. Sigh. A key strategy for me is to convince the business of what needs to be tested where, oterwise… they’ll end up directing testing everything throug the UI… So the two key main points I am making to my business are the need for test automation that performs well in terms of two key factors:
Success in both speed and reliability for software test automation is hard and most technology companies struggle with both.
The overall goal of software test automation is to provide faster feedback to the business at all levels from the development team to the various levels of management, appropriately summarized at each level.
For software test automation itself, using the UI to create test data or set application state is incredibly slow and in some cases not even possible and will thus not provide the timely feedback that is required.
This is frequently a 10x+ (and often 100x+) factor in terms of testing speed and dramatically affects the ability to both develop and test automation. If automation takes 20 minutes to reach a specific spot in a workflow instead of 7 seconds this will make quality automation development impractical. Consider also then trying to run that same test in 20 different device-browsers when that means 20×7 = 2+ hours for that 7 second test. Imagine wanting to try 10 different changes to get it right (common).
and the agile testing quadrants
to ensure getting the benefits of swift feedback throughout the the development process.
There are two key technical problems to solve:
- setting up test conditions and state without using the UI
- controlling test and application state through APIs
Traditionally the need for test automation has not be enough of a critical business success factor to justify actually spending serious application development effort, time and money on addressing these two key technical issues (they are certainly solveable with enough effort and determination). That has now changed and test automation is increasingly recognized at a hugely critical busines success factor underpinning so many other efforts and plans of todays businesses.
For the detailed strategy of dividing up unit, integrated tests and UI tests, a few thoughts come to mind?
- Create some examples of functionality with (e.g.) 60 unit, 12 integrated and 4 UI tests to show the test pyramid in action
- Make sure unit tests mock and stub dependencies
- Ensure that appplication and automation engineers share code reviews of each others work
- Expose application environment stability for functional tests constantly in the application/automation team space area
- Expose test status information constantly in the application and automation development area
- Ensure that application developers write units tests and use code coverage and quality measurement tools as a required part of their continuous integration code development process.
- Ensure that application developers and UI automation developers are physically close to each other for easy drop-in chats.
- Consider creating a formal triage system (e.g. during backlog refinement) to determine what to test where (in terms of unit, int and uat tests) at least until you can do it instinctively.