When it comes time to test a new release of a software application, I'm always anxious to dig into the new functionality to start mapping out exactly what needs to get poked, prodded, and kicked. Figuring out how to test what's new is, of course, going to be interesting because, well, it's new. On the flip side, there's also regression testing that needs to be performed to make sure that everything that used to work still works. In between the two there's some integration testing that makes sure the new functionality plays nice with the old functionality (and vice versa).
New functionality testing will always entail creating new test scenarios, test plans, and data sets so you know you have to dedicate the time and effort to do it. Regression testing... well, that's mostly done already, right? It's easy to leave your regression scripts sit, untouched, until it's time to execute them since you're testing things that shouldn't have changed, right? It's especially easy to have this mindset in a smaller organization, but if you do give your regression scripts even a small amount of focus before re-using them, maybe updating them just a tad, you can make your regression so much more effective.
First, just do a scan through your regression scripts to make sure none of the new functionality has changed the existing functionality. If it has, make the appropriate modifications to your regression scripts so that they don't break down (if automated) or introduce confusion (if manual). I've run into more than a few situations where I had to modify regression scripts while running through them because something had changed.
Second, and I find this most important when reviewing your scripts, take a quick stock of where you had previously identified defects in releases past and make sure to cover situations that deal with replicating or attempting to reproduce those defects. You'd be surprised at how many times you'll find new defects in the same area or, heck, even notice that the previously uncovered defects aren't completely resolved. If you had an area of the application that had a few defects, pay special attention and make sure your regression scripts are giving the proper attention to that area--there are probably more issues where those you already found came from.
Third, look at where you can eliminate some testing. Let's be honest here--regression testing takes a long time, especially if you got a little overzealous when you wrote your scripts. You might not need to go through all of the tests you have in your scripts. Look for places where you can combine test cases or test steps. On one project I was able to combine 3 separate test chunks into a single battery of tests. I had separate testing chunks for adding a specific piece of data, deleting that type of data, and migrating that data. They were all written at different times so were in different spots, but as I reviewed the overall script, I found I could combine those tests into one test script that created the data, then migrated that created data, and finally deleted it. Combinations of testing chunks might not be as obvious as that, but if you know your application well and have a keen eye for evaluating your test scripts, you should usually find a spot or two where you can make them more efficient.
Lastly, and this should be obvious but it doesn't always get done, make sure to add regression testing for functionality that was added in the previous release. It's easy to be lax about maintaining your scripts and not adding to them with each release, but it is essential that you have scripts for new functionality from the last release. If you don't, you leave yourself with a gap that could become quite dangerous. You're going to give plenty of focus to your new functionality and your unmodified regression scripts will cover most of the older existing functionality, but the newer existing functionality may receive no testing which could leave defects uncovered.
I know regression testing isn't sexy and isn't always fun, but it needs to be done. Since it needs to be done, why not make it as efficient and effective as possible? It's not that difficult to give a little bit of focus to your scripts, so overcome that urge to assume you don't need to touch those scripts and give them the tune-up they need.
No comments:
Post a Comment