A vision for our testing future

As I eluded to in my previous post, it's time for us as a community to fully embrace the place of automated testing in our workflows. Over the past year we've learned how to write testcases and to then apply that knowledge towards writing autopilot tests. At the same time ubuntu has been building testing infrastructure and launching a CI team to help run and maintain it.

With these changes and our acquired knowledge we can construct a sustainable vision for testing. So just what is that vision?

Let's make manual testing fun again. As we've ramped up our workloads, we find ourselves repetitively testing the same packages again and again (and running the same tests!). We'd call this regression testing and it's a perfect candidate for automated testing. Let's automate these tests and keep our focus on where we as humans excel.

In addition, we as a community participate in "calls for testing". These calls have become more and more exploratory in nature. In general, we have moved further away from a defined testcase ("perform steps 1, 2, and 3"). Instead, we encourage testers to adopt a try and break it attitude. This is where we as humans can excel, and you know what, have some fun at trying! Remember QA is the only place we encourage you to break things!

So let's get manual testing back to the exploratory testing we love.

Thank you little testing robot!

Let's expand our automated tests. We can increase testing coverage forever with a one time investment of writing a testcase. Developers, write tests for your code and feature sets as you develop them. As a quality community, let's do our best to make it easy for developers to do so.

In addition, a well written bug report can become an excellent testcase to be added to the application testuite. Let's look at bug reports for potential testcases. We can write the test and then forget it. The little robots will tell us if it becomes a problem again in the future and even prevent the bad code from getting shipped where it can break our machines (again). Not bad little robots!

So let's focus on ensuring tests exist for critical regressions when they get fixed.

Let's tackle some big projects. Who said we can't achieve amazing things in the quality community? This vision would be nothing more than an idea without some actions behind it!

Right now we as a community already have a chance to put these ideas in action. Exploratory testing is happening right now on the phablet devices. Get involved! This is the manual, have fun and try and break it, exploratory testing we all love. Even without a phablet device, regressions can be turned into autopilot tests. This meets our goal to expand our automated tests and to look at bugs as potential sources for those tests.

All test and no play makes for a sad image

Moving forward there are still a some big projects to tackle. One of the largest is the amount of manual testing effort required in cadence and milestone testing. We can bring automated testing to the mix to both reduce our on-going effort and raise the quality bar in our releases.

So let's be the leaders for change in ubuntu.

Are you in?