Tuesday, July 31, 2012

A video walkthrough of ISO Testing

Notice I'm not mentioning a good video walk-through, but it is in HD! It can be viewed on youtube here. I had a lot of trouble doing my first screencast, but I think I have a promising setup now for the future. I promise HD, lag-free, and better editing in the future. For reference, I highly recommend flowblade to edit and kazam to record. Kudos to both developers for nice pieces of software!

The text version is available here and I encourage you to follow along with the video. You don't need anything more than a reasonably modern pc that can run a VM and you can help iso test. Take 20 minutes to watch and follow along and take the fear out of iso testing. Then join us next week as we continue our testing cadence and verify our daily isos are in good shape.

And don't worry, no one will know it's running the installation on a separate workspace while you watch youtube videos.. It'll be our secret.

Wednesday, July 25, 2012

What can webapps do? Help test and see!

As I'm sure you've already heard, webapps are becoming first rate citizens on ubuntu. Some of you might even have installed the ppa and are enjoying exploring what webapps does for you on the desktop. If you've been one of those early adopters and are wondering exactly what all webapps can do, or are wondering how you can help squash bugs, this is your chance!

The webapps team has made available some manual testcases that cover the functionality found in webapps currently. These testcases have been made available via a 'call for testing' for webapps. Head over to the package tracker and select the tests corresponding to your version of ubuntu. For precise, Webapps Precise Testing, and for quantal Webapps Quantal Testing.

Never participated before in a call for testing? No worries! Check out this handy walk-through and you'll be submitting results in no time. Thanks for helping test!

Tuesday, July 24, 2012

Remembering the good ole days

Quality has been a buzzword for a couple releases now. Certainly, it's safe to say we saw many more people talking about quality last cycle than any previous cycle I can remember. Ubuntu of course has done LTS's in the past, but something about this past release was different. People yearned again for the perceived quality of the past LTS release and wanted to see ubuntu succeed.

Ahh yes, the good ole days of 8.04 or 10.04 LTS releases when everything was right in the world and ubuntu just worked, etc. Heh, it's easy for us to remember the past and the good things we enjoyed. But to the extent those releases served me well, I have the community to thank. Ubuntu is community, and it's success or failure is determined by all of us.

Open source projects are not typically known for there quality. In the same way ubuntu has innovated and paved the ground for a consumer focused desktop linux, I believe we as a community are uniquely positioned to show how open source can be better quality than competing ideological offerings. In the same way a minuteman can win a battle against a better equipped mercenary, so too can a dedicated community provide better quality software than a commercial offering. This only makes sense. I would rather work with a group of passionate people than with folks who don't care about there work.

One of the best parts about getting to work in QA is seeing the end result of the entire communities work. Those who perform QA represent the last pair of eyes for the work our community does. The developers, translator, doc writers, bug triagers, forums and IRC admins can all do marvellous work in support of ubuntu. But if it doesn't install/work on your pc, then sadly you won't be able to enjoy any of that work.

Recently, those running the development version of ubuntu noticed a bug that caused there webcam to no longer work. The community responded by helping figure out exactly what commit caused the regression. Many folks were involved here! People with the issue tested different kernel versions to narrow down when the regression occurred, while the kernel team made these kernels available and provided insight. Ultimately with everyone's help the problem code was identified and confirmed by testing a custom built kernel. Armed with this knowledge an upstream bug was created.

Now as I sit here today, my webcam works again! And not just for me, or other ubuntu users, but for everyone who uses the linux 3.5 kernel. After upstreaming the bug,  the original developer made a fix available which we tested. Ultimately, this fix made it into the 3.5 kernel. This prevented the 3.5 kernel from shipping with broken logitech webcams for everyone. This is the power of our community!

The story is but one of many examples of problems the ubuntu community has solved. If you enjoyed or remember the good ole days in ubuntu, I would encourage you to get involved. If your running ubuntu on your desktop now and want to continue doing so, consider donating your skills to help. A healthy QA community is vital for ubuntu to continue to grow and get better. We, as a community, can be the standard for quality in open source.  So how can we take ubuntu, and specifically quality in ubuntu, to the next level? It starts with you.

Monday, July 16, 2012

Testing in Cadence

Last month, an interesting thread emerged on ubuntu-devel. A proposal to change the way we as ubuntu look at testing and quality. In many ways it was more of a codification of ideas and thoughts from the precise cycle than a proposal. 

One of the outcomes of this was a change to how to test isos. Rather than focus on arbitrary moments in time, we've been asked to stick to a two week cadence for testing. What that means is a regular checkup of our images every two weeks. Quite a task, but not impossible! Given the fact the change happened mid-cycle, there has been some confusion over what exactly this means. I decided to put together a post detailing exactly what's on the table for us as a community and more importantly how you can help!


If you have a look at this wiki page, I've listed the images we test and produce for ubuntu. So far during the quantal cycle we have achieved 100% coverage for most of these iso's for our mandatory testcases. That's thanks to the wonderful efforts of folks like you testing isos! But in some cases, like our first non-milestone cadence last week, the coverage was provided by a single person -- meaning we have only 1 confirmation of success or failure. I'd like us as a community to take this to the next level. I'm asking for you to commit to an iso over the course of the cycle. Would you be willing to commit to running through the mandatory testcases every 2 weeks for the iso and making sure it's in good shape? If so signup on that wiki page, underneath the iso in question. Don't be afraid if this sort of testing sounds scary. I and the rest of the community are happy to help you through your first testcase. As part of iso testing, I'm still growing my knowledge of linux and ubuntu and interacting with wonderful and talented people while doing it. This is a natural expansion of the 'adopt an iso campaign' with a new cadence. Powerpc, and mac users, this is your chance to make a difference as your hardware is less common and therefore harder to ensure proper testing for.


During the precise cycle over 100 people submitted a result to the isotracker for an iso they downloaded and tested as part of daily testing. That's excellent work, and I thank all of you very much! Many more of you downloaded and installed iso's throughout the cycle, but perhaps didn't report your work. I would encourage you to get involved and help share your results with others. If we have 100 people signup in support of iso testing, the workload required of each individual will be quite small. Yet the benefits for us as a community will be huge. More hardware and more testing results in more bugs caught sooner. We all want a good upgrade experience in October. This is your chance to be a part of making sure it happens.


As a small addendum, I'd like to point out the results of the work this testing achieves. If you have a look on this page, you'll notice a very long list of bugs; many of which are rated as high or critical in launchpad. All of these bugs were found during iso testing -- a testament to those who have tested before us. We all thank you.

Monday, July 2, 2012

Call for Testing: 12.10 kernel on 12.04 -- Part Deux

As announced earlier the kernel team is looking for a some folks to help bring the 12.10 kernel to 12.04. Once 12.10 has been released, the team wants to enable newer hardware support via the kernel for the LTS version of the desktop. So, since the original announcement, we've had 10 people help test the various builds of the kernel from the ppa. Thanks so much to all who tested! Now we'd like to take this testing to the next level.

We've put together a list of commonly used hardware that we want to ensure proper support for the kernel. I'm asking for volunteers to run the kernel from the ppa on precise and report results. The catch here is that we want to have at least 1 person who has each piece of hardware listed be represented. Make sense? For example, we want folks running nvidia cards to have at least a couple people reporting results using nouveau and the proprietary driver. Same for AMD and Intel. On the wireless side, getting someone who has a chipset of each of the manufacturers listed is our goal. This is a first step in our on-going efforts to help make testing and quality a more assured and quantitative effort. We're going 'quantal' if you will. For those who want to have even more detailed and specific hardware testing, hang tight. If you'll remember this past UDS we spoke about creating a community hardware database. Work to enable this is on-going, and I hope to be able to share more about it in the coming months. In the meantime, let's build up a list of folks and systems ready to populate such a database, shall we? ;-)

So if your interested in helping, go ahead and edit that wiki page. Add yourself under one or more pieces of hardware. There's a handy script that should help you identify what's in your system if your not quite sure. Then head over to the QATracker. 


Once there click on 'Quantal kernel for precise LTS', you will wind up on a page showcasing the tests and instructions for this call for testing. If you click the on 'Link to the installation information' you will get information on installing and uninstalling the package. Similarly the 'Link to bug reporting instructions' provides details on reporting a bug you find in the test case. Finally, if you click 'Kernel Smoke Tests' you'll arrive on the page to report your testcase results. Note you will need to sign in using your ubuntu sso account to report results.

If you encounter issues, you can always reboot into your current working kernel and be back to normal. As always, if you have any issues in using the tracker, feel free to get in touch with me.