Preparing for the Community Charity-a-thon

In preparation for the 24 hour marathon, I thought I would share with everyone my thoughts on my I chose my charity, and what I plan to do for 24 hours :-) To start off the post, let me get this right out front <insert flashing lights and whirlygigs> DONATE HERE </insert>

First the charity, WaterAid. I chose WaterAid upon realizing how important water is to me. I love water -- I love to stare out across a vast sea, or to sail along it using the wind, and hearing only the splashing of the waves, and the smell of life beneath it. And of course, I consume water each day in order to sustain life. I am happy to support an organization whose goal it is to provide sustainable water sources to everyone. Water is important to life, and is a basic need for us all as humans. We need clean water, and even moreso, we need sustainable access to it. Water is precious, and it's important for us to not pollute the water we have as well. WaterAid understands this need and works with locals to help create clean renewable water sources. Consider donating to help those who don't have access to the same resource we take for granted -- available anytime out of our faucet.

I'm also placing a call out to those who are interested in QA on both the Debian and ubuntu communities to participate. I plan to spend my time during the 24 hours doing something to further the work of how you interact with ubuntu and QA. So, to that end, I'd like to ask those of you who are interested in ubuntu to donate and install ubuntu during the marathon. I'll be here to provide technical installation support for you during the install. Let's see how many exotic configurations we can see successful installations on. Got a RAID setup or some exotic hardware? Multiple disks, partitions, and Os's? Get your systems backed up now, and let's try and install. NOTE, I'd encourage using the latest daily iso for installation, but you are welcome to also use beta2.

In addition for those of you in the Debian community, I am issuing a challenge for donations. For every 5 donations from the Debian community, I will write a missing man page from the list. I'll be focusing on things I use, but feel free to offer a suggestion during the marathon!

I would also issue a challenge to the greater ubuntu community. Do you have a problem that you are unable to solve within ubuntu? While I can't promise a fix for your issue, I will offer you my personal attention to help solve your problem. I'll help you file a bug, confirm it, or help you debug the problem to the best of my ability. I'll even offer my karma on askubuntu to your question ;-) If I get overwhelmed with donations, I'll pull the highest donators first -- but we do have 24 hours to fill! Note, I plan to do this work on Thursday, on-air, but you can donate in advance. Just leave me a note, or simply send me an email after your donation with your request if you donate in advance.

Finally, if none of the above suits you, I am happy to have a personal 1 on 1 match of highest level house of cards building, or another quick playing game of your choice on air. And don't worry, I'm ruthless competition, no pushover here!

Getting your bug fixed; the art of bug shepherding

We've all had this experience. Upgrade our hardware to the new version of ubuntu (or perhaps installing ubuntu for the first time on some new hardware), and everything works perfectly. We quickly dive in and enjoy the new experience of our favorite OS. Well, it's almost perfect. There's just this one bug that is bothering you.

Perhaps instead you are a tester. Someone who lives and breathes for breakage and the latest and greatest raw code packaged from the hands of it's developers. There's a new upgrade out and everything is perfect; well, except for that one bug.

So what do you do in these situations? Sadly some people may chose to do nothing but hope and wait. "Next cycle, or next release I'm hoping my bug will be fixed". Odds are, no one knows about your bug, and it can't be fixed if it's unknown. And if you are using proprietary software, "wait and see" is about the limit of your options. As a part of ubuntu however, we as a community can do so much more!

With that let me present my patented wizzbang approach to a successful resolution of your bug within ubuntu!

  
First, let me clarify what a successful resolution is. If you have been around ubuntu, you may have seen a bug that expired as 'incomplete'. This is clearly not a successful resolution. In my eyes, a successful resolution to a bug sees the status changed to a 'won't fix, fix committed, triaged, etc'.

Ok, so here's the steps:

1. If you don’t know how to file a good bug, ask first! It's important to do your best to describe the problem you are experiencing, and if possible how to repeat the bug. Check out the post on askubuntu which has a nice summary of resources available to help you.
2. File a good bug, using your newly formed knowledge from above :-)
3. Get someone else to confirm it. This is important! If possible, have a friend confirm the bug on their system. Once they've confirmed it, have them mark the bug as affecting them as well on launchpad.
4. Answer questions promptly when asked by others. Make sure you are getting emails from launchpad, and when someone asks a question on your bug, respond promptly.
5. Get your bug triaged. If your bug is confirmed and filed correctly, the bug triagers should help triage the bug. If a long time has passed without this occurring, check to make sure you bug is in good order to be triaged. If so, asking politely for a triager to look at your bug on the #ubuntu-bugs channel is a good way to keep your bug moving.
6. Help debug, test, and confirm and fixes that are made for your issue. If the developer spends time working on your bug, do what you can to help confirm it fixes your issue.
7. Remember no one will care about your bug as much as you do! When you file a bug, commit to carrying it as far along in the process as you can.

That's it! There's no guarantee every bug you file now will receive your desired outcome, but you should see proper resolution, instead of your bugs expiring. By being a good participant you are ensuring you can feel good about the resolution of the bugs you file. Remember we are all human, and sometimes things get missed. Stick with your bug, and shepherd it through.

So is the process perfect? Not at all. We as a community still need to think more about improving our experience in dealing with problems. Not every "problem" encountered is a bug, and a process to better handle these problems is still worthy of thought. I invite those of you interested in this to look for a UDS session on the topic.

Special thanks to TheLordofTime and hggdh for their discussions surrounding bugs, and of course for our marvelous bugsquad without whom this would not be possible!

Global Jam with QA love

Global Jam is a unique time for folks to get together across the world and celebrate ubuntu. As part of the celebrations, there is an opportunity to download the latest beta (released yesterday!) and check out the next version of ubuntu. You can run it in a livecd or perhaps in a virtual machine. Ether way, there's opportunities for you to test the common applications and report your results.

The testcases are available here: Ubuntu Global Jam Testcases. For some of you, this page may seem a bit funny, but fortunately there is a handy walk-through for you to understand how to use the tracker and specifically, how to report results against these tests. Check out the guides below. If you follow the link, you'll even find some video of yours truly giving you a visual demonstration.

https://wiki.ubuntu.com/Testing/QATracker
https://wiki.ubuntu.com/Testing/CallforTesting/Walkthrough

If you get stuck, remember your friends in #ubuntu-testing on freenode are also happy to help. Have fun jamming, and if you do test, Happy Testing! Most of all, celebrate ubuntu!

Ubuntu QA goes social

Since I posted the survey results, some of the members of the qa community noticed the team was unknown by others in the greater ubuntu community and beyond. After posting the article and ending my day, no sooner do I awake then my inbox already contains links to several social media locations where groups have been setup. I was floored and impressed by everyone's excitement to let others share in the fun we have here in QA (great work everyone!).

So, what does that mean for everyone else? We've gone social! Check out the new groups, and if your already a member of the social community in question, I would encourage you to join up if your interested in qa. And for anyone who has been involved in the oppurtunities we undertake in qa, I would encourage you to write about them. Share them with your friends.. in a word, communicate :-) You don't need to find a new medium, take qa and ubuntu to where you already interact!

https://wiki.ubuntu.com/QATeam/Contact

Call for Testing: Compiz & Unity

It's the testing event I've been waiting for! A new version unity has arrived, stock full of compiz fixes. The team has removed metacity completely, and also migrated to gsettings from gconf. The result is removal of unity2d, and the enablement of llvmpipe on unity for those running without hardware acceleration.

Now, of course this work, in particular the settings migration, needs to be tested. This is where all of you come in! The package tracker now contains an entry for unity, complete with testcases for this migration. For those who helped test last cycle, you will also notice all of the checkbox testcases have been ported over as well! Many thanks to the unity developers for there help in migrating these tests. For this call for testing, the 'Unity GSetting Migration' testcases have been marked as mandatory, meaning that testcase is the primary focus. However, if you are able, executing the other testcases also helps the unity team ensure there haven't been any regressions.

Please note the 'Unity GSetting Migration' has steps for you to complete BEFORE you install from the ppa. Please read it first before diving in. Here's a link to the testing on the tracker. And if you are new, check out our wiki for a guide on using the qatracker to help test.

Now, as an added bonus, the unity developers have also tagged several multi-monitor bugs and have asked users to go through the list and confirm any bugs that they can. Read the bug report, and if you have a multi-monitor unity setup, see if it's still affecting you. Leave a comment on the bug with your result. The unity team wants to make sure the bugs are able to be triaged and get proper attention if they are still valid.

List of unity bugs, tagged multi-monitor

Thank you in advance for your help, and happy testing everyone!

The grand "Cadence" experiment

It all started innocently enough. A simple idea, turned into a simple post on the mailing list. This idea eventually led the ubuntu QA community to perform an experiment for this cycle, which has been dubbed "cadence testing".

Now, before I manage to confuse everyone with this "cadence testing" term, let's define cadence testing. Scratch that, let's just give a simple definition of what was intended by original idea. If you want the whole story read the thread. heh. I'll be waiting (hint it's LONG!).

Cadence testing was intended to introduce regularity into testing. If the development release could be "stable" everyday (which was the grand experiment during the precise cycle), could we not also test to ensure that things were good all throughout the release? If the everyday images and archive were now to the quality of previous release's milestones, could we just eliminate the milestone idea and go with a calendar schedule for testing? Thus, a proposal was made test every 2 weeks, whether or not a milestone had been planned, and report the results.

Fast forward 2 months to today. So what happened? Well, I'm happy to report that the QA community despite the confusion more or less met the goal of testing the desktop images every 2 weeks (milestone or not). But what did this achieve? And where are the results?

Let's step back a moment and talk about what we learned by doing this. My comments are specific to the non-milestone cadence testing weeks. First, the development process inside ubuntu is still built around milestones. The daily images during cadence testing weeks were sometimes stable, and sometimes flat out broken by a new change landing from a development team. Second, the tools we used are built around milestone testing as well. The qatracker as well as any qa dashboard or report available doesn't have a good way to track and image health across the cadence week. This meant it was both difficult to test and difficult to see the results of the testing. Finally, the development teams were not expecting results against the daily images, and couldn't follow-up well on any bugs we reported, nor where we able to coordinate well with the release team, as the bugs reported were not available in a summarized or meaningful way.

Now, I'll save the discussion on my ideas of a healthy QA workflow for a later post, but I think we can agree that testing without good result reporting, and without developer follow-up has a limited impact. So does this mean "cadence testing" was a bad idea? No, it was simply poorly executed. The trouble comes in the assumptions listed above.

The archive has not been "stable" everyday, and development teams have have continued development, pausing only as required by the current milestones. In addition, changes, even major ones (like the ubiquity changes landing a few weeks ago, or the nvidia change just this past weekend), are not well communicated. Since they land without little or no warning, we as a QA community are left to react to them, instead of planning and executing them. In this environment, cadence testing makes little sense.

So was the experiment a failure then? In my mind, not at all! In fact, I think the future of ubuntu and QA is to push for complete adoption of this idea, and this experiment confirms the obstacles we will face in getting there. I'll be posting more about what this vision for QA looks like, but I'll leave you with a few thoughts until then.

In my mind, QA should enhance and improve developers, testers, and users lives and workflows. Our work is critical to the success of ubuntu. I would like to see a future where users receive regular, timely scheduled updates that the folks in the QA community have vetted by working with the development and release teams to deliver focused quality updates. The ideal workflow is more fluid, more agile and yes, it has a cadence.

Quality Perceptions Survey Results

A couple Fridays ago I asked for feedback on how quality and the ubuntu QA team this cycle. That survey has now been completed and I have some results to share with everyone. Before I dive into the numbers, let me take a moment to say thank you to all of you who responded. Thank you! I read all of the comments left as well, and all were helpful feedback. Remember the survey was anonymous, so I cannot respond individually to anything written. Feel free to contact me if you wish to discuss anything further or to receive a response.

The second question on the survey asked rather simply, "What does quality mean to you?".

As it turns out, the largest answers mirrored those of a later question, in which I asked "What's the biggest problem with quality in ubuntu right now?".

Note, I read all of the "other" responses and categorized them into some new categories to display here for general consumption.

So there is some agreement amongst those who were polled both about what quality means, and about where ubuntu's biggest problems lie. The respondents indicated the largest issue with quality in ubuntu, according to them, was also the definition of what "quality" is!

Now I asked this question for a specific reason. "Quality" is a subjective term! Perhaps I'll get some disagreement on this, but hear me out. All of the answers for the question in my mind are valid with respect to quality. As an example, let's say, I asked you to recommend software to balance my checkbook. If I specified I wanted a quality piece of software, would you not recommend to me a stable (works without crashing), mature (good development/bug workflow), and easy to use (just works) piece of software that has a nice feature set (latest and greatest)? It's easy to see that "quality" can refer to all of this and more.

Still, in my mind, when I speak to wanting a "quality" release of ubuntu, I tend to focus on the stability and ease of use aspects. As the graphs indicate, the respondents seemed to echo this idea. In other words, it's really important to the idea of quality that things "just work". In the context of ubuntu this means applications run without crashing, and the operating system runs on your hardware. If things don't "just work", even if all the other indications of quality are true, you aren't likely to describe or perceive the product as having "good quality".

Let's ponder that thought for a moment and look at some more results. The survey captured about a 50/50 split of folks who run the development release, and over 70% run it or intend to run it before the final release.

So among those 50-70% who run or will run the development release, how many have participated in ubuntu qa?

Yikes! Only about a third. Just under half have no idea a ubuntu QA team existed. There's some clear evangelizing work to be done here. Let me take pause here just for a moment to say the team does exist, and would love to have you!

Ok, now onto the last multiple-guess multiple-choice question.

I'm happy to see people desire to help! That's wonderful. The responses regrading time, being technically able, or where to start are all very solvable. I would encourage you to watch this space for invitations to help test. QA work comes in all shapes and sizes, sometimes it's as little as 15 minutes, and the ability to install/uninstall a package and reboot a machine. If this sounds like something you would be able to do, please start by having a look at our wiki page. Send us an email and introduce yourself. There's no requirements or forced participation and we welcome everyone. And who knows, you might even learn something about ubuntu :-)

Ok, so I've shared the hard numbers from the survey, but I'd like to leave you with a few takeaway thoughts. First, while quality is subjective, our focus in ubuntu QA should be to have things "just work". That doesn't mean we shouldn't also help improve our development and bug processes, or continue to push for new applications and features, but rather that we ensure that our efforts help forward this cause.

I've said it before, but I want to help deliver good computing experiences. That story I shared when I introduced myself was close to home. My first interaction with the ubuntu community came via the forums, and yes, getting a printer to work. The community undertook work to change what was once a nightmare not for the feint of heart to child's play. The execution of this work is what defines the experience. This is where QA fits. We aren't just testing; we're delivering the result of the entirety of the ubuntu community's labor.

Judging from the survey results, many of you share this same vision. So won't you join us? QA transcends across teams and the ubuntu community. I would encourage you to get involved and be a part of making it happen. The list of "problems with quality" reach many areas. Would you be part of the solution?