Testing all the things: Terminal App

I couldn't help but start with one of the core apps I consider essential (to me anyway!) on my phone, a terminal. The terminal app being developed for ubuntu has some wonderful features built with a touch interface in mind. One of the biggest issues with touch is having a terminal ready keyboard with things like page up and down, arrow keys, and not to mention being able to use keyboard shortcuts like ctrl+d, ctrl+z, ctrl+c, etc. This has been handled rather elegantly with a long tap menu as you can see below, in addition to a panel that optionally appears at the top of the application.

Dmitry Zagnoyko has already landed a few tests for some of the features present, as you can see below. Execellent work Dmitry! A basic testcase exists now for each of the panels and the circle menu.

Help Dmitry and the terminal app team make sure all the features work properly for you upon release. Get involved and add a test. The initial setup work has already been done, and there are existing testcases already written. Grab the terminal branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge.

Testing all the things: Introduction

As a quality community team, we've been continuing to make progress this cycle on automating our testcases, especially the new applications that are being written for ubuntu touch. These 'core apps' are being written by other community members for the next generation of ubuntu.

We're also making progress on our desktop applications and automating the ubiquity installer. With that in mind, I'm going to start a little blog series highlighting a package a day for automating. I'll dub it rather unoriginally "Testing All The Things". My goal is to showcase the wonderful work going on with testing this cycle in ubuntu, but also to encourage you dear reader to get involved in helping us. All areas of ubuntu (flavors too!) can benefit from some robot friends helping test the packages they work on and utilize.

But you don't need to wait to see your favorite app hit the list. Hit up the tutorials below for information to dive in and help us!


Core Apps Test Wiki
Writing an autopilot test for ubuntu sdk applications
QML Autopilot Tutorial with example application

Autopilot Tests Project
Writing an autopilot test for desktop applications 

Update on automated testing with autopilot

Given all recent love and excitement for autopilot I wanted to share the QA community's progress on writing autopilot tests for, celebrate our successes and let everyone know where we still need help.

First let's talk about the ubuntu-autopilot-tests project. As part of the hackfests held at the end of May/early June we were able to complete the transition to autopilot 1.3 of the ubuntu desktop autopilot tests. Thanks to all of the contributors and hackers for helping on this! In addition, we now have a production branch, and the canonical platform QA team is working on adding them to the official smoke testing each day, Great work everyone! That said, tests are still needed, and in some cases the testcases are still basic and not covering many of the application features. There is still room for you to be invovled! Of note is the on-going work to automated our image testing via the UI.

Next, let's talk about the core apps. Last Thursday we held a hackfest to help kickstart testcases for all of these projects. So let's take a look at how far we've come in a week. As a reminder, testcase contributions to any of the core apps is very much appreciated -- there is still a need for you to come alongside and help write tests!

Calculator
There are already several testcases merged in with the main branch, but as one of the most feature complete applications, work and help is still needed in this area. There are currently 6 open bugs for tests needed here. This is a great application to contribute to for someone new to autopilot!

Calendar
There are two pending merge requests and the work is underway towards knocking out the rest of the testcases needed.

Clock/Alarm
The clock team has jumped in headfirst to help with testcases.  You can view the status of the remaining tests needed here.

Doc Viewer
I started on a branch for this and the basic infrastructure is in place. Branch the application. Grab a copy of the emulator, pick a bug and write your test. This app needs you!

File Manager
The first merge and test is in review. But there's still more tests to be written. Have a look at the list of needed tests.

RSS Reader
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!

Terminal
The first merge request has just been approved and landed for terminal autopilot tests. But there's more features to be tested in this awesome app. Grab something off the list and go. The setup work is already done.

Music
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!

Weather
Half of the initial testcases have been started and the first merges are being proposed. Rock on Martin!

Remember you can always view the big master list of all the open tests here. We've got a bit of work ahead of us! Be a part of the team. Grab an open bug from the list above or contact me for help and I'll make sure you get invovled!

An autopilot emulator for ubuntu sdk apps

A couple weeks ago we announced the initiative to drive up our autopilot (that is, automated) tests for our ubuntu touch core apps. The core apps are being made with the ubuntu sdk, and thus share the same language (QML) and toolkit (ubuntusdk).

With this in mind I wanted to provide an emulator, which in autopilot speak, is a utility class for writing autopilot tests that use the ubuntu SDK. The goal is to help accelerate the process for getting the testcases written, as well as standardizing best practices for testing common features. At the moment the emulator contains useful functions like tab switching, selecting from popovers, opening and closing the toolbar and clicking toolbar buttons. Please, take a look and utilize the emulator when you are contributing new tests for the ubuntu touch applications. For the moment, the emulator can be found here:

lp:~nskaggs/+junk/ubuntusdk_autopilot_emulator

The future home is hopefully in the SDK itself, but for now consider that branch your source for emulator goodness. Now, a quick FAQ.

Is it ready for use?
Yes, it's ready and tested on several core apps now including clock, calendar, terminal, and file manager. That said if you find an issue, simply contact me or propose an improvement!

How do I use it?
Inside your autopilot test subfolder, add an emulators folder if it's not already present. Next, branch my source above -- it will add ubuntusdk.py to the folder. Simply incorporate it into your __init__.py or testcase itself and call the utility functions with ubuntusdk.*. For an example check out the ubuntu-terminal-app and the merge request from today. It shows adding autopilot tests to an empty branch. In addition, the emulator (albeit an earlier version) was used in the tutorial on the ubuntu app developer portal.

Will it be updated?
Yes! Expect refinements and tweaks as we go along. Hopefully a true "stable and complete" version will appear in the not too distant feature when the emulator itself has a proper home. In the meantime, use it and as more complex tests are added, expect to update the emulator in the source branch you are working in.

Go forth and write tests!

QATracker Survey + bonus mockup

Hot on the heels of our first cadence week, I wanted to take the opportunity to collect feedback about the tools we as a community utilize. Specifically the QATracker which we heavily rely on for managing our work, testcases and results. From the wiki, "The QATracker is the master repository for all our our testing within ubuntu QA. It holds our testcases, records our results, and helps coordinate our testing events."

This is a link to a brief survey asking a few simple questions about how you've used the tool. All your responses are anonymous, but I will publish the aggregate question information and share it with the community once completed. The goal is to help ensure the tool is meeting our needs and is being utilized.

I'll leave the survey up until June 24th. My hope is to encourage more folks to help test as well as make it more enjoyable for those already taking part. I want to ensure our tools and processes continue to evolve, strengthen and become more robust for everyone as we continue on our mission. Part of that is making sure the tools we use are enjoyable!

Thanks in advance everyone!

As a bonus, Pasi, aka knome, has put together some mockups on how we might be able to switch what the results page looks like. This is perhaps the most utilized page of the site, so without further ado, here's a mockup of some changes proposed to make it more usable:

Old Site
New Site Mockup

What a change eh? The add test results has been moved to the sidebar and simplified, the bugs listing has been written out, and the results have been moved to the top. Finally the links have also been moved to the sidebar and Pasi has updated the icons ;-)

SO, what does everyone think about the changes? Many thanks to Pasi for putting this together! Leave a comment, a message on the mailing list, or reflect your thoughts in the survey.

Feeling Saucy? Help us Test the new release!

Join the ubuntu quality community team's effort this week! As a community we test different things about every ~2 weeks in ubuntu, and share the results to flesh out bugs and problem areas.

So what's up for testing this week? The daily images, the default applications in ubuntu and a new version of the sound stack for testing.

Ready to help? Full details are here.

Need some help on how to contribute? Have a look at this page and the walkthroughs listed. Of particular interest is the ISO testing and Cadence Week testing walkthroughs.

Do note that you don't need anything special to participate in cadence week testing! Both an installed version of the development branch of ubuntu (aka saucy) in a VM or on a real box, or even a live session of the latest daily image will work. For more information on how to use a live session to test, check out the Cadence Week testing walkthrough or watch the youtube video of the same.

Happy Testing!

Autopilot Tests for Ubuntu Touch Core Apps

A few months ago the ubuntu touch core apps project was launched. For those of you following along with Michael's regular updates have gotten to see these applications grow up rather quickly.

Autopilot Says: How can I help?
Now it's time to add some more testing around these applications as they have reached a basic functional level of usability. Automated testing via autopilot to the rescue!

To help kickstart this process we've put together a recipe for writing autopilot tests specific to QML applications and added it to developer.ubuntu.com. In addition, we'll be hosting a hackfest next week on June 13th to help add basic autopilot testcases for each of the core apps. Folks will be on-hand ready to field your questions and hack together on the autopilot testcases needed for the applications. Join us and help support the wonderful community of application developers making awesome applications for ubuntu!

So how can you help? 

1. First, go read through the recipe on writing autopilot tests for QML applications. It's also a good idea to have a look through the official tutorial for autopilot and bookmark the API reference link so it's handy.
2. Armed with your new knowledge, start hacking on some autopilot tests for the core apps. Here's a list of core applications along with the status of autopilot tests. Choose something that looks interesting to you and add some tests.
3. Follow the contributing guide to help you get your work contributed into the ubuntu touch core application project you chose.
4. Finally come out to the hackfest! It's your chance to share your work, ask questions, get your tests sorted and merged and socialize and meet other members of the community.
5. Don't forget there is a wonderful quality community you can be a part of and get help from if you get stuck! There's a mailing list for ubuntu-touch, and ubuntu-quality as well as IRC channels #ubuntu-touch, #ubuntu-autopilot and #ubuntu-quality. Use these resources to help you!

See you next week and happy testing!