Time to test trusty!

Say that three times fast. Time to test trusty,
time to test trusty, time to test trusty!

Ahh it's my favorite time of the cycle. This is the part were we all get serious, go a little bit crazy, and end super excited to release a new version of ubuntu into the world. This time it's even more special as the new version is a brand new LTS, which we look forward to supporting for the next 5 years.


The developers and early adopters have been working hard all cycle to put forth the best version of ubuntu to date. For you! For all of us! It's time to fix bugs, do last minute polish and prepare for the release candidate which will occur around April 11th.

We need you!
This is were you dear reader come in. You see despite their good looks and wonderful sense of humor and charm, the release team doesn't like to release final images of ubuntu that haven't been thoroughly tested.

The release team is ready to pounce on untested images

We need testing, and further, we need the results of that testing! We need to hear from you. Passing test results matter just as much as failures. The way to record these results is via the isotracker; we can't read your mind sadly!

How to help
Mark your calendars now for April 11th - April 16th. Pick a good date for you and plan to download and test the release candidate image. You'll see a new milestone on the tracker, and an announcement here as well when the image is ready. I won't let you forget, promise!

Execute the testcases for ubuntu and your favorite flavor images. Install or upgrade your machine and keep on the lookout for any issues you might find, however small.

I need a guide!
Sound scary? It's simpler than you might think. Checkout the guide and other links at the top of the tracker for help.

I got stuck!
Help is a simple email away, or for realtime help try #ubuntu-quality on freenode. Here's all the ways of getting ahold of the quality team who would love to help.

Community
Plan to help test and verify the images for trusty and take part in making ubuntu! You'll join a community of people who do there best everyday to ensure ubuntu is an amazing experience. Here's saying thanks, from me and everyone else in the community for your efforts. Happy testing!

Trusty looms closer: Final Beta is here!

It may be hard to believe but the next, and dare I say, best, LTS for ubuntu is releasing very soon. We need your help in polishing out critical bugs and issues!

How can I help? 

To help test, visit the iso tracker milestone page for 'Trusty Beta 2'.  The goal is to verify the images in preparation for the release. Find those bugs! The information at the top of the page will help you if you need help reporting a bug or understanding how to test. 

So what's new? 

Besides the usual slew of updates to the applications, stack and kernel, unity has new goodies like minimize on click, menus in toolbar, new lockscreen, and borderless windows!

What if I'm late?

The testing runs through this Thursday March 27th, when the the images for beta 2 will be released. If you miss the deadline we still love getting results! Test against the daily image milestone instead.

Thanks and happy testing everyone!

Keeping ubuntu healthy: Core Apps

Continuing our discussion of testing within ubuntu, today's post will talk about how you can help the community core apps stay healthy.

As you recall the core apps go through a series of QA before being released to the store. However bugs in the application, or in the platform itself can still be exposed. The end result is that the dashboard contains tests failures for that application. To release a new stable image, we need a green dashboard, and more importantly we need to make sure the applications work properly.

Getting plugged in
So to help out, it's important to first plug into the communication stream. After all, we're building these applications and images as a community! First, join the ubuntu phone group on launchpad and sign up for the phone mailing list. The list is active and discussing all issues pertaining to the ubuntu phone. Most importantly, you will see landing team emails that summarize and coordinate issues with the phone images.

From there you can choose a community core app to help improve from a quality perspective. These applications all have development teams and it's helpful to stay in contact with them. Your merge proposal can serve as an introduction!

Finding something to work on
So what needs fixing? A landing team email might point out a failing test. You might notice a test failure on the dashboard yourself. In addition each application keeps a list of bugs reported against it, including bugs that point out failing tests or testing needs. For example here's the list of all new autopilot tests that need to be written for all of the core apps. Pick an app, browse the buglist, assign a bug to yourself, and fix it.

For example, here's the list of bugs for music app. As of this writing you can see several tests that need written, as well as a bug for a test improvement.

You can also simply enhance the app's existing testsuite by fixing a flaky test, or improving the test to use best practices, etc. As a bonus for those reading this near it's original publication date, we just had a session @ vUDS covering the core apps and the testing needs we have. Watch the session / browse the pad and pick something to work on.

Fixing things
Look into any failures you find and have a look at the tests. Often the tests can use a little improvement (or maybe an additional test), and you can help out here! Sometimes failures won't happen every run -- this is the sign of a weird bug, or more likely a flaky test.  Fix the test(s), improve them, or add to them. Then commit your work and submit a merge proposal. Follow the guide on the wiki if you need help with doing this.

Remember, you can iteratively run the tests on your device as you work. Read my post on click-buddy for help with this. If you are lacking a device, run the tests on your desktop instead and a reviewer can test against a real device before merging.

Getting Help
For realtime help, check out #ubuntu-quality and #ubuntu-autopilot on freenode. You'll find a group of folks like yourself working on tests, hacking on autopilot and sharing advice. If IRC isn't your thing, feel free to contact us through another method instead. Happy hacking!

Keeping ubuntu healthy: Manual Image Testing

Continuing our discussion of the testing within ubuntu, today's post will talk about how you can help ubuntu stay healthy by manually testing the images produced. No amount of robots or automated testing in the world can replace you (well, at least not yet, heh), and more specifically your workflow and usage patterns.

As discussed, everyday new images are produced for ubuntu for all supported architecture types. I would encourage you to follow along and watch the progression of the OS through these images and your testing. Every data point matters and testing on a regular basis is helpful. So how to get started?

Settle in with a nice cup of tea while testing!

The Desktop
For the desktop images everything you need is on the image tracker. There is a wonderful video and text tutorial for helping you get started. You report your results on the tracker itself in a simple web form, so you'll need a launchpad account if you don't have one.

The secondary way to help keep the desktop images in good shape is to install and run the development version of ubuntu on your machine. Each day you can check for updates and update your machine to stay in sync. Use your pc as you normally would, and when you find a bug, report it! Bugs found before the release are much easier to fix than after release.

Phablet
Now for the phablet images you will need a device capable of running the image. Check out the list. Grab your device and go through the installation process as described on the wiki. Make sure to select the '-proposed' channel when you install so you will see updates to get the latest images being worked on every time they are built. From there you can update everyday. Use the device and when you find a bug, report it! Here's a wiki page to help guide your testing and help you understand how and where to report bugs.

Don't forget there's a whole team of people within ubuntu dedicated to testing just like you. And they would love to have you join them!

A simple look at testing within ubuntu

Since just before the last LTS, quality has been a buzzword within the ubuntu community. We've come a long way since precise and I wanted to provide some help and prospective on what ubuntu's process for quality looks like this cycle. In simple terms. Or as reddit would say, "explain to me like I'm 5".

I'll try and define terms as we go. First let me define CI, which is perhaps the buzzword of this cycle, lest I lose all of you! CI stands for continuous integration, and it means we are testing ubuntu. All the time. Non-stop. Every change we make, we test. The goal behind this idea is to find and fix problems, before well, they become problems on your device!

CI Dashboard
The CI dashboard then is a way to visually see the results of this testing. It acts as a representation of the health of ubuntu as a distribution. At least once a day runs are executed, apps and images are tested and benchmarked, and the results are populated on ci.ubuntu.com. This is perhaps the most visible part of the CI (our continuous testing efforts) that is happening within ubuntu. But let's step back a minute and look at how the overall CI process works within ubuntu.

CI Process
App developers hack on a bit of code, fixing bugs or adding new features to the codebase. Once the code is ready, a merge proposal₁ is created by the developer and feedback is sought. If the code passes the peer review and the application's tests, it will then become part of the image after a journey through the CI train.

For the community core apps, the code is merged after peer review, and then undergoes a similar journey to the store where it will become part of the image as well. Provided of course it meets further review criteria by myself and Alan (we'll just call him the gatekeeper).

Though menacing, Alan assures me he doesn't bite

Lest we forget, upstream₂ uploads₃ are done as well. We can hope some form of testing was done on them before we received them. Nevertheless, tests are run on these as well, and if they pass successfully, the new packages will enter the archive₄ and become part of the image.

Generating Images
Now it's time to generate some images. For the desktop a snapshot of what's in the ubuntu archive is taken each day, built, and then subjected to a series of installation tests. If the tests pass, it is released for general testing called Image (or ISO) testing. An image is tested and declared stable as part of a milestone (or testing event) and can become the next version of ubuntu!

Adopted images are healthy images!

On the ubuntu phone side of things, all the new uploads are gathered and evaluated for risk. If something is a large change, it might be prudent to not land it with other large changes so we can tell what broke should the image not work properly. Once everything is ready, a new image is created and is released for testing.  The OTA updates (over-the-air; system updates) on your ubuntu phone come from this process!

How you can help?
Still with me I hope? As you can see there's many things happening each day in regards to quality and lots of places where you can create a positive change for the health of the distro! In my next few posts, I'll cover each of the places you can plug in to help ubuntu be healthy everyday!

1. A merge proposal is a means of changing an applications code via peer review.
2. By upstream, I mean the communities and people who make things we use inside of ubuntu, but are not directly a part of it. Something like the browser (firefox) and kernel are good examples.
3. This can happen via a general sync at the beginning of the cycle from debian. This sync copies part of the debian archive into the ubuntu archive, which in effect causes applications to be updated. Applications are also updated whenever a core ubuntu developer or a MOTU uploads a new version to the archive. 
4. In case you are wondering, "the archive" is the big repository where all of your updates and new applications come from!