There are a number of systems that can handle this. I’m familiar with Link and TFS, both of which have APIs (although the TFS API is a little challenging – speaking as someone who’s worked out how to post results to TFS through the API) and both of which have ways to handle cases that need to change.

A lot depends on your budget – Test Link is free (as in open source, and as in beer), TFS is not (Microsoft Team Foundation Server, if you’re not familiar with the abbreviation). QA Complete from SmartBear is another commercial tool that offers the functionality you’re looking for.

Here’s how these tools (and I suspect the other major commercial offerings) work:
You define one or more test environments – this covers your device types, browsers, OSs and so forth. Depending on the tool you define the environments required at the test suite, test plan, or test case level. The API to communicate the tool varies, but can be worked with. For TFS it needs a fair amount of code and assorted other dependencies taken care of before you start the tests. Test Link also has dependencies that need to be set up in the tool before you start.

For test case updates, the usual arrangement is that the test case is versioned once it has been run (that is, there’s a result posted to the tool). The rationale is that if the test case has to change, the historical results will be invalid unless you retain the history of the test case itself. Updating changed test cases is usually a matter of creating a new version (and incrementing the test case versioning), then running against that until there’s another change.

Depending on the tool, you may be able to break down your test cases into reusable steps that minimize the amount of re-versioning you need to do.

I know Team Foundation Server allows this through what it calls “Shared Steps” – which are pretty much what they sound – one or more steps in a test case which can be used by multiple test cases. (Caveat – Team Foundation Server needs to use Microsoft Test Manager for have test suites and test plans defined – currently it’s not possible to create these through the TFS interface – and MTM can’t be used without TFS).

I’ll often set up a single step as a shared step so it can be used by any test case (such as “click the logon button” which doesn’t care what the user credentials are). You can also parameterize steps so that instead of a whole lot of steps for each different value your tests will enter into a particular field, you have one step to enter “@value” into the field, with the values that can be used defined by the test case or an attached data file.

Test Link doesn’t have that level of reusability, that I know of. I don’t know enough about other tools to offer any suggestions.

There is a good listing of test tools available in the Test section of the Ministry of Testing’s test tool listing. (It’s a long list… you’ll need to do a fair amount of research in this)

On the plus side, most of the commercial tools do offer some form of trial. I know Microsoft has a free service for TFS that they host (Team Foundation Service) and contains the most up-to-date feature set. That’s free for up to 5 users per account – I don’t know any other details.

Good luck finding something that meets your needs.

Source link


Please enter your comment!
Please enter your name here