Trusty Cycle Updates

First things first, we have our burndown up for the trusty cycle! If your name appears on the list next to a work item, congratulations! Let's work through the items and make it happen ;-) If your name doesn't appear, don't worry! Feel free to help complete the tasks anyway, or jump in an get started on something you love. The mailing list is always open and we love to hear new ideas. See a blueprint that looks interesting and want to help? Get in touch!

What's happening now?
Now, for an update as to what is going on in quality. This week starts Alpha 1 for the flavors that are participating. In addition, the first work items are starting to see completion.

Ali has been working on getting the wiki pages and workflow ready for our community calls for testing. You can check out the page on the wiki here, and schedule testing events. The page explains the basic workflow -- I'd encourage you to schedule events for your favorite packages and let others know about the event on the mailing list. Be sure and report the results once you've completed.

Alberto has been working on getting the papercuts project running for the trusty cycle and ensuring the buglist is maintained and accessible.

Dan has been working on getting our automated testing in place and we're meeting with the CI team this week to get this implemented as part of the release process. Excellent work Dan!

Dan has also been working on the autopilot gtk emulator, which seeks to do the same thing as we've done with the autopilot ubuntu sdk emulator. Both of these frameworks help make writing tests easier.

Speaking of tests, Carla and others have been busy adding and fixing tests to make sure our dashboard stays green. And look at it! We're over 99% passing tests for the ubuntu touch platform. I want to personally thank everyone who's helped write and maintain these tests. Kudos to you all!

On the desktop side, Dan is helping new folks land tests, and the ubuntu desktop default apps continue to run and provide feedback on the desktop.

We also saw several new members join the ubuntu community via ubuntu quality recently. Congrats to Jean-Baptiste, Dan Chapman and Daniel Kessel (my apologies if I missed you!). 

Finally we have Thibaut and other folks looking at refactoring and improving ubuntu startup disk creator. Checkout the full blueprint for more information.

So, what's next?
Well for most of us, it's the Holiday season. Looking beyond the Holidays, , there's several big items in my list to work on in January:

1. Continue to keep the dashboard green all the time
2. Do wiki, mailing list, launchpad integration with bugsquad
3. Add wiki documentation and videos for exploratory testing

What am I doing?
Well, you dear reader, I trust is getting involved in some of the work items and other activities. It starts with installing the development version of ubuntu and then diving in from there. Perhaps you want to write some tests, triage or fix some bugs, or schedule a testing event. If you are interested in making a contribution to ubuntu, we can always use good testers, test writers, developers and triagers!

If you are celebrating holidays this time of year, enjoy your time!  Thanks everyone and Happy Testing!

Quality and the community in the Trusty Cycle

Since I last posted on the topic of our communities future, we've helped push out a new release of ubuntu. Saucy and fresh, the little salamander is now out in the wild. With the release out it's time to move forward with these changes.

(c) Carla Sella

The beginning of a new cycle is always a time to breathe for us in quality and these past few days of relative calm has indeed been a welcome respite from the craziness of testing leading up to a release.

We can rebuild it. We have the technology. Better than it was before. Better, stronger, faster.

So, let's talk about some of the changes coming to quality for this new cycle.

Defined Roles
I want to help our new members become a productive part of ubuntu quality as soon as possible. To this end I've created a list of roles for the quality team. By defining roles within the team it is easy to see how you can contribute as a 'tester', 'test writer', or 'developer'.

Contribute anytime
While release milestones and calls for testing will still be important, contributing to ubuntu quality can be a daily task. There are activities you can perform any day and anytime in the cycle. The roles pages list activities for each role that can be done right now. If you are a tester, check out the activities you can do right now to help!

More exploratory testing
We're iterating faster and faster. Builds of new code are landing each day, and in some cases several times a day. We can't afford to only test every other week with cadence testing. Instead, we've ramped up efforts to automatically test on each of these new builds. But we still need manual testing!

As a tester you can provide testing results for images, packages and hardware at any time! In addition, exploratory testing is highly encouraged. This is were we as manual testers shine! I want to encourage ongoing exploratory testing all throughout the cycle. Run and use the development version of ubuntu on your machine all the time!

Tackling some big projects
One of the things I wanted to push us as a team towards was tackling some projects that have a wider impact than just us. To that end, you can see several big projects on the activities pages for each role.

For testers, we are undertaking making reporting issues better for users. For the test writers, one of the largest projects is spearheading the effort to make manual image testing less burdensome and more automated by automating ubiquity testing. And for developers, the autopilot emulators are big projects as well and need help.

More involvement with bugs
As a quality community with interact with many bugs. Sometimes we are finding the bug, other times we might confirm them or verify a fix works. However, we haven't always gone the extra step towards doing work with SRU verifications and bug triage. The bugsquad and others have traditionally performed these tasks. If you'll notice these activities are now encouraged for those in the tester role. Let's dive more into the bug world.

A potential expansion of the team
With the mention of the bugsquad and the encouraged involvement with bugs for our team, I would like to propose a union between the quality and bugsquad teams. I would encourage current bugsquad members to take up tester roles, and consider some of the additional opportunities that are available. For those who have been testing with the quality team in past cycles, let's get more invovled with bugs and traditional bugsquad activities as mentioned above.

Making a quality LTS
Trusty Tahr is going to be the next LTS release. Those of you who remember and use Precise will agree that it is going to be a tough act to follow. The bar is set high, but I am confident we can reach it and do better.

We can't do this alone. We need testers, test writers, and developers! If you are interested in helping us achieve our goal, please join us! Now is an excellent time to learn and grow with the rest of us on the team. Thanks for helping making ubuntu better!

Testers, assemble! Final RC testing is here!

(c) http://www.flickr.com/photos/lalalaurie/

Here are 3 easy steps to testing prowess. Do your part to be prepared! Let's help make saucy a great release!

1) Review both the critical bugs and those found during isotesting throughout the cycle. This is handy in case you find something during testing and want to know if someone has already reported it or not.

2) Make sure you know how to use the tracker and how to test an iso. When the milestone is created and the images appear on October 10th, you want to be ready to go!

3) Test and report your results to the tracker ;-) Note, at this point in the cycle we prefer real hardware results. Upgrade your machine! Try a fresh install. If you've been sitting on raring, now is the time to migrate to saucy and let us know of any bugs you find.

Happy Testing!

Dancing, testing, and releasing

We're still here and we're still testing!

And you can still join us!

Grab your nexus device, check out the wiki page, and flash away. File bugs, and keep ontop of the updates coming out. Use the device as you would any other phone. Ohh and check out the app development contest winners by installing the apps from the click store on the phone (check out the more suggestions section).

The release is only 2 weeks away. That's 14 days or a mere 336 hours. And we'll be spending 112 hours sleeping (perhaps less, get some sleep!). That leaves just 224 hours for testing. Join us!

#UbuntuTestDanceMaster out.

A vision for our testing future

As I eluded to in my previous post, it's time for us as a community to fully embrace the place of automated testing in our workflows. Over the past year we've learned how to write testcases and to then apply that knowledge towards writing autopilot tests. At the same time ubuntu has been building testing infrastructure and launching a CI team to help run and maintain it.

With these changes and our acquired knowledge we can construct a sustainable vision for testing. So just what is that vision?

Let's make manual testing fun again. As we've ramped up our workloads, we find ourselves repetitively testing the same packages again and again (and running the same tests!). We'd call this regression testing and it's a perfect candidate for automated testing. Let's automate these tests and keep our focus on where we as humans excel.

In addition, we as a community participate in "calls for testing". These calls have become more and more exploratory in nature. In general, we have moved further away from a defined testcase ("perform steps 1, 2, and 3"). Instead, we encourage testers to adopt a try and break it attitude. This is where we as humans can excel, and you know what, have some fun at trying! Remember QA is the only place we encourage you to break things!

So let's get manual testing back to the exploratory testing we love.

Thank you little testing robot!

Let's expand our automated tests. We can increase testing coverage forever with a one time investment of writing a testcase. Developers, write tests for your code and feature sets as you develop them. As a quality community, let's do our best to make it easy for developers to do so.

In addition, a well written bug report can become an excellent testcase to be added to the application testuite. Let's look at bug reports for potential testcases. We can write the test and then forget it. The little robots will tell us if it becomes a problem again in the future and even prevent the bad code from getting shipped where it can break our machines (again). Not bad little robots!

So let's focus on ensuring tests exist for critical regressions when they get fixed.

Let's tackle some big projects. Who said we can't achieve amazing things in the quality community? This vision would be nothing more than an idea without some actions behind it!

Right now we as a community already have a chance to put these ideas in action. Exploratory testing is happening right now on the phablet devices. Get involved! This is the manual, have fun and try and break it, exploratory testing we all love. Even without a phablet device, regressions can be turned into autopilot tests. This meets our goal to expand our automated tests and to look at bugs as potential sources for those tests.

All test and no play makes for a sad image

Moving forward there are still a some big projects to tackle. One of the largest is the amount of manual testing effort required in cadence and milestone testing. We can bring automated testing to the mix to both reduce our on-going effort and raise the quality bar in our releases.

So let's be the leaders for change in ubuntu.

Are you in?

Spreading the (testing) weight

With all the buzz and excitement around quality over the last few cycles, we've been busy. We've built tools, started processes, wrote testcases and committed ourselves to testing. When the time came for us to write new applications for the ubuntu touch platform, we took our quality aspirations with us and tested all through development. The "core apps" for the phone is a perfect example of this; community developed software including tests. Kudos to everyone who is taking part in these efforts.

However, as more and more tests came online, new problems developed. How can we run these tests? When do we want to run them? How can we develop more? What tools do we use? The quality team met all of these challenges, but the weight started to grow heavier.

Rob Macklem Victoria BC derivative: Plasticspork [CC-BY-SA-3.0]

Something needed to be done. Before the bar breaks or our arms give out, let's split the weight. And thus a new team was formed.

Ivanko Barbell Company

Enter the CI team. Behind the scenes running tests, getting our infrastructure in place, and chasing down regressions, the CI team has their hands full. In a nutshell they will ensure all the tests are run as needed. If a test fails they will chase down and ensure the right folks are notified so fixes can be made. Ohh, and yes, that means potential bugs will never even hit your system dear user.

The QA team's mission then gets to be refocused. They will continue to write tests, develop tools if needed, and continue to push for greater quality in ubuntu and upstream. However they no longer have to balance these tasks with ensuring builds stay green, or writing a dashboard to review results. The result is a better focus and the opportunity to do more.

So how does this affect us as a community? The simple answer is that we too can take advantage of this new testing infrastructure and CI team. Let's focus on what coverage we need and what the best way to achieve it might be. We'll also be able to align our goals and ambitions even more with the QA team. I'll share more of my thoughts on this in the next post.

Autopilot Sandboxing

Autopilot has continued to change with the times, and the pending release of 1.4 brings even more goodies; including some performance fixes. But today I wanted to cover a newly landed feature from the minds of Martin and Jean-Baptiste (thanks guys!).

If you've developed autopilot tests in the past you will have noticed how cumbersome it can be to run the tests. If you run a test on your desktop, you lose control of your mouse and keyboard for the duration, and you might even accidentally cause a test to fail. This can be especially noticeable when you are iterating over getting your test to run "just right" while wanting to keep your introspection tree in vis open, or reviewing someone else's code while wanting to verify the tests run. Enter sandbox mode.

A new command called autopilot-sandbox-run lets you easily run a testsuite inside your choice of two sandboxes; Xvfb by default, or if you want to visually see the output, Xephyr. Have a quick look at the command options below as of this writing:

Usage: autopilot-sandbox-run [OPTIONS...] TEST [TEST...]
Runs autopilot tests in a 'fake' Xserver with Xvfb or Xephyr. autopilot runs
in Xvfb by default.
   
    TEST: autopilot tests to run

Options:
    -h, --help           This help
    -d, --debug          Enable debug mode
    -a, --autopilot ARG  Pass arguments ARG to 'autopilot run'
    -X, --xephyr         Run in nested mode with Xephyr
    -s, --screen WxHxD   Sets screen width, height, and depth to W, H, and D respectively (default: 1024x768x24)

The next time you want to get your hands dirty with some autopilot tests, try out the sandbox. I'm sure you'll find a very nice use for it in your workflow; after all wouldn't it be handy to run multiple testsuites at once?

Testing Ubuntu Touch: The final month before release

As of today, we are exactly one month away from the release of Saucy Salamander. As part of that release, ubuntu is committed to delivering an image of ubuntu-touch, ready to install on supported devices.

And while folks have been dogfooding the images since May, many changes have continued to land as the images mature. As such, the qa team is committing to test each of the stable images released, and do exploratory testing against new features and specific packagesets.

If you have a device, I would encourage you to join this effort! Everything you need to know can be found upon this wiki page. You'll need a nexus device and a little time to spend with the latest image. If you find a bug, report it! The wiki has links to help. Testing doesn't get anymore fun than this; flash your phone and try to break it! Go wild!

And if you don't own a device? You can still help! As bugs are found and fixed, the second part of the process is to create automated tests for them so they don't occur again. Any bug you see on the list is a potential candidate, but we'll be marking those we especially think would be useful to write an autopilot tests for with a " touch-needs-autopilot" tag.

Join us in testing, confirming bugs, or testwriting autopilot tests. We want the ubuntu touch images to be the best they can be in 1 month's time. Happy Testing!

Jamming Quality Style

It's that time of year again! Time to get your jam on (I like mine on a bagel).

  

While you are making plans for Ubuntu Global Jam, don't forget you can contribute to quality as well. There's a separate subpage of the global jam wiki dedicated to it.

We love new test contributions, and there's a collection of wiki tutorials and videos to help you contribute them. You don't have to be technical to write tests -- we need manual testcases also which are written in plain English :-)

More interested in submitting your results for tests? We've also got you covered. We have tests for the default applications of ubuntu as well as the images of ubuntu. Download an image and run it on your machine. Try running through some default testcases for ubuntu or your favorite flavor. An image and a pc or laptop is all you need to get started. Happy Jamming!

Call for Testing: Mir with multi-monitor

As mentioned in my last post, Mir is one of the biggest changes coming in 13.10. With feature freeze now happening this week, it's time to amp up our testing engines once more to test the final features and help land Mir into the archive.

The Mir team has put together both a ppa and wiki page that contains all the information you need to help with testing. The testing window closes in 2 days on August 28th, just in time for feature freeze. The biggest changes for Mir are the inclusion of multi-monitor support and thus are a focus for this testing. So here's the details you need to know.

What?
Help test Mir using your current system, ubuntu saucy and the Mir team ppa.

When?
Now through August 28th.

How?
The full instructions for installing the ppa, running the tests, and reporting the results can be found on this wiki page. Results are reported on this page or via the package tracker testing page.

Thank you for your contributions! Good luck and Happy Testing Everyone!

Automated Testing in ubuntu

So with all the automated testing buzz occurring in the quality world of ubuntu this cycle, I wanted to speak a little about why we're doing the work, and what the output of the work looks like.

The Why
So, why? Why go through the trouble of writing tests for the code you write? I'll save everyone a novel and avoid re-hashing the debate about whether testing is a proper use of development time or not. Simply put, for developers, it will prevent bugs from reaching your codebase, alleviate support and maintenance burdens, and will give you more confidence and feedback during releases. If you want your application to have that extra polish and completeness, testing needs to be a part. For users, the positives are similar and simple. Using well tested software prevents regressions and critical bugs from affecting you. Every bug found during testing is a bug you don't have to deal with as an end user. Coincidentally, if you like finding bugs in software, we'd love to have you on the team :-)

The How
In general, three technologies are being used for automated testing.
Autopilot, Autopkg, and QML tests. Click to learn more about writing tests for each respectively.

Any color is good, as long as it's green

The Results
The QA Dashboard
The dashboard holds most of the test results, and gives you a much nicer view of the results than just looking at a jenkins build screen. Take some time to explore all of the different tabs to see what's available here. I wanted to highlight just a couple areas in particular.

• Autopkg Tests 
• These testcases run at build time and are great testcases for low level libraries and integral parts of ubuntu. Check out the guide for help on contributing a testcase or two to these. Regressions have been spotted and fixed before even hitting the ubuntu archives.
• Smoke Tests for ubuntu touch
• This is some of the newest tests to come online and displays the results of the ubuntu phone image and applications, including the core apps which are written entirely by community members. Want to know how well an image is running on your device? This is the page to find it.

Ubiquity Installer Testing
Curious about how well the installer is working? Yep, we've got tests for those as well. The tests are managed via the ubiquity project on launchpad.

Ubuntu Desktop Tests
Wondering how well things like nautilus, gedit and your other favorite desktop applications are doing? Indeed, thanks to our wonderful quality community, we've got tests for those as well. The tests can be found here.

The Next Steps
We want to continue to grow and expand all areas of testing. If you've got the skills or the willingness to learn, try your hard at helping improve our automated testcases. There's a wide variety to choose from, and all contributions are most welcome!

Sorry robot, all our tests are belong to us

Don't have those skills? Don't worry, not only are machines not taking over the world, they aren't taking over testing completely either. We need your brainpower to help test other applications and to test deeper than a machine can do. Join us for our cadence weeks and general calls for testing and sample new software while you help ubuntu. In fact, we're testing this week, focusing on Mir.

Finally, remember your contributions (automated or manual) help make ubuntu better for us all! Thank you!

Feature freeze coming? Let's test!

We're already approaching feature freeze at a quickening pace, and thus the next few weeks are rather important to us as a testing community. 13.10 is landing in October, which is now rapidly approaching (where did the summer go?!).

What?
Let's run through some manual tests for ubuntu and flavors. I'd like to ask for a special focus to be given to Mir/xMir. We plan to have a rigorous test of the package again in about a week once all features have landed. In the interim, let's try and catch any additional bugs.

When?
This week Saturday August 17th through Saturday August 24th. It's week 5 for our cadence tests.

Ok, I'm sold, what do I need to do?
Execute some testcases against the latest version of saucy; in particular the xMir test.

Got any instructions?
You bet, have a look at the Cadence Week testing walkthrough on the wiki, or watch it on youtube. If you get stuck, contact us.

Where are the tests?
You can find the Mir test in it's own milestone here. Remember to read and follow the installation instructions link at the top of the page!
The rest of the applications and packages can be found here.

I don't want to run/install the unstable version of ubuntu, can I still help?
YES! Boot up the livecd on your hardware and run the live session, or use a virtual machine to test (install ubuntu or use a live session). The video demonstrates using a virtual machine booting into a live session to run through the tests. For the Mir/xMir tests, however, we'd really like results from real hardware.

But, virtual machines are scary; I don't know how to set one up!
There's a tool called testdrive that makes setting up a vm with ubuntu development a point and click operation. You can then use it to test. Seriously, check out the video and the walkthrough for more details.

Thank you for your contributions! Good luck and Happy Testing Everyone! 

Autopilot best practices

I've now had the pleasure of writing autopilot tests for about 9 months, and along the way I've learned or been taught some of the things that are important to remember.

Use Eventually
The eventually matcher provided by autopilot is your best friend. Use it liberally to ensure your code doesn't fail because of a millisecond difference during your runtime. Eventually will retry your assert until it's true or it times out. When combined with examining an object or selecting one, eventually will ensure your test failure is a true failure and not a timing issue. Also remember you can use lambda if you need to make your assert function worthy.

Assert more!
Every test can use more asserts -- even my own! Timing issues can rear there ugly head again when you fail to assert after performing an action.

• Everytime you grab an object, assert you received the object
• You can do this by asserting the object NotEquals(None); remember to use eventually Eventually(NotEquals(None))!
• Everytime you interact with the screen, try an assert to confirm your action
• Click a button, assert
• Click a field to type, assert you have focus first
• You can do this by using the .focus property and asserting it to be True
• Finished typing?, assert your text matches what you typed
• You can do this by using the .text property and asserting it to be Equal to your input

Don't use strings, use objectNames
We all get lazy and just issue selects with English label names. This will break when run in a non-English language. They will also break when we decide to update the string to something more verbose or just different. Don't do it! That includes things like tab names, button names and label names -- all common rulebreakers.

Use object properties
They will help you add more asserts about what's happening. For instance, you can use the .animating property or .moving property (if they exist) to wait out animations before you continue your actions! I already mentioned the .focus property above, and you might find things like .selected, .state, .width, .height, .text, etc to be useful to you while writing your test. Check out your objects and see what might be helpful to you.

Interact with objects, not coordinates
Whenever possible, you should ensure your application interactions specify an object, not coordinates. If the UI changes, the screen size changes, etc, your test will fail if your using coordinates. If your interaction is emulating say something like a swipe, drag, pinch, etc action, ensure you utilize relative coordinates based upon the current screen size.

Use the ubuntusdk emulator if you are writing a ubuntusdk application
It will save you time, and ensure your testcase gets updated if any bugs or changes happen to the sdk; all without you having to touch your code. Check it out!

Read the documentation best practices
Yes, I know documentation is boring. But at least skim over this page on writing good tests. There is a lot of useful tidbits lurking in there. The gist is that your tests should be self-contained, repeatable and test one thing or one idea.

Looking over this list many of the best practices I listed involve avoiding bugs related to timing. You know the drill; run your testcase and it passes. Run it again, or run it in a virtual machine, a slower device, etc, and it fails. It's likely you have already experienced this.

Why does this happen? Well, it's because your test is clicking and interacting without verifying the changes occurring in the application. Many times it doesn't matter, and the built in delay between your actions will be enough to cover you. However that is not always the case.

So, adopt these practices and you will find your testcases are more reliable, easier to read and run without a hitch day in and day out. That's the sign of a good automated testcase.

Got more suggestions? Leave a comment!

Automating the Core Apps: How we doing?

This month has been all about dogfooding, and part of that has been the drive to automate the testing of the functionality found in the core applications. Nothing will replace true user testing, but we can certainly do a lot to help prevent bugs and regressions. So, during my "test all the things" series we went through each of the core applications and highlighted areas for testing. So the question is, how are we doing as this month draws to a close?

Well, fortunately, I've setup this wiki page as a reflection of where we stand.
As you can see, we've come a long way on many of the applications, but I wanted to especially point out where we still need help.

Dropping Letters
Adrian has been working on Dropping Letters and could use some help in finishing the testcases. Leo has proposed a merge that brings the testsuite up to date with the new sdk, so it's ready for you to add tests!

RSS Reader
RSS Reader, aka shorts, just went through some large UI changes and is now ready for us to add tests again. Carla has begun working on updating her old tests for adding and editing a feed, but there's more work to do!

Calendar
Calendar is currently undergoing some drastic changes, and it's testcases will need to be re-written once that's complete. Hang tight if your keen to help in this area!

Music
Music has come a long way in a couple weeks and we have tests to prove it! Thanks to Daniel and Victor, we can test play, pause and library load. But there are still more tests needed. John is also volunteering on this application to help get the tests in shape. Thanks guys!

The following applications are very close to completion:

Clock
Nekhelesh has hacked together most of the tests, but I know he won't complain if you help him finish. Review the list and help bring it over the line!

Calculator
As of this morning, the final calculator testcase has been written, testing tear-off. Pending a bugfix in the sdk, the testcase will get merged. Fingers crossed. Thanks Riccardo!

100% DONE!
Congratulations to the following teams and folks who helped us reach 100% status on the following core apps:

terminal, weather, sudoku, stock ticker, file manager

Remember though, as we move forward it's important to keep the tests aligned with new features :-)

Still reading this? What are you waiting for? Jump in and help! Happy Automated Testing!

An autopilot emulator for ubuntu sdk apps: Part II

A little over a month ago, I posted about creating an autopilot emulator for
ubuntu sdk applications. Thanks to some hard work by Leo Arias and the ubuntu sdk team, I'm happy to announce the little side project from my +junk branch is all grown up and even more ready for your consumption. Leo has made the emulator a proper part of the sdk, complete with tests for all of it's functions!  You can now easily install it and incorporate it into your project. Here's how to snag your copy.

1) Add the ubuntu sdk team ppa if you haven't already

sudo add-apt-repository ppa:ubuntu-sdk-team/ppa

2) Install the ubuntu-ui-toolkit-autopilot package

sudo apt-get update && sudo apt-get install ubuntu-ui-toolkit-autopilot

This will install the emulator as a python module, ready for you to import. If you want to checkout what the module can do, have a look at the documentation.

Incorporating the module might seem a little tricky, so Leo has also put together an example of one of the ubuntu touch core apps using the new module. Check out it. Here's a branch showing off the work done for ubuntu-filemanager-app. And here's one for dropping-letters.

Please do check out the module and incorporate it into your ubuntu sdk project. Feedback is encouraged, and bug reports too! Please file any issues you find against the ubuntu-ui-toolkit project. Happy Automated Testing!

Ubuntu Edge: Convergence in the palm of your hand

Several years ago I was having a conversation with a bunch of colleagues of mine, most of whom would not be described as linux or ubuntu advocates. Seeing the launch of the tablet era with the iPad I lamented on having a truly portable pc in my pocket. We talked at length about the idea of a converged device. Within a few years we've seen android tablets ship (I was truly thrilled to see the transformer series come out) and phones like the atrix supporting webtop mode with ubuntu.

Still, even with a new android phone in my hand I want something a bit more. I don't enjoy carrying around heavy laptops, so a few years ago, I stopped when given the opportunity. I finally got rid of my old CRT from 1997 (yes, you read that right, RIP old friend) and got a nice widescreen display to replace it. But I'm still running the same tired old desktop, with my phone sitting next to it on the desk. It's certainly powerful enough to do everything I need, and if not, I've got a cloud server for long running computational tasks. So, why have the tower?

Why not carry my phone with me as my pc? Throw it on my desk and I can use my big screen, mouse and keyboard. At the train station I can use it with it's built in screen, but still have local and remote access to everything I need. Then perhaps while seated on the train I use it in something more akin to a traditional laptop since I have the room.

Well, this idea of a convergent device is here, and it's called ubuntu edge. A device that looks like a traditional phone but has pc-like specifications. It runs the same applications and OS as a computer. And it supports multiple form factors for interacting and using the applications. This idea of convergence is the biggest story for me.

So, Check out the campaign page on indiegogo. Watch the video that shows off the beautiful OS and applications we're building to go with the amazing hardware. That's right, "we" the community are building the applications. Now it's our chance as a community to help build an amazing piece of hardware and run them.

Testing all the things: Music

The music app. A vital and necessary piece of my running setup. I've run in silence before, and while it's nice, sometimes you need some jams to keep you motivated on longer runs.

Recently I've been running in silence but not by choice, definitely needing a way to play music via my phone. Thanks to the hack day for music last week and the ongoing work on the music app developers that's all changing. Today the app is feature complete enough to begin work on some autopilot tests. It can for instance, play music now!

Consider helping the music app developers keep development going.  Grab the music app branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, there is a wonderful video tutorial to help you. In addition the logs, and a FAQ from the workshops are now available and ready for you to consume and understand. Happy test writing!

Core Apps Burndown: Ramping up for success

Today I thought we'd take a break from the testing all the things series to take a look at what we're doing, how far we've come, and most importantly, where we need to get to.

The core apps project has come a long way since it's inception at the beginning of this cycle. So as the development started maturing, testing was the natural step for me to get involved and check out what the apps had to offer. With that in mind, let's look at the burndown chart for the core apps shall we?

Can you tell where the tests were added?

So even as the work items has increased from adding things like tests, this month the community have been working hard at finishing work items and bringing us back to the trendline. See that nice driving downwards the last couple weeks? Excellent job! Let's get down to that trendline!

From the quality side, we've had several people come forward, learn about autopilot and the core apps and then get a merge in. A special thanks to Carla Sella, Daniel Kessel, Michael Spencer, Adrian Goodyer, Riccardo Padovani and Arturas Norkus for your contributions last week to core apps projects. Well done, and I look forward to seeing and merging more of your work!

For those of you still working on getting commits in, or sitting on the sidelines, let me help! You can be a part of this effort! There's still more work to do, and the entire community around core apps would appreciate your help. Hack on an app, a testcase, wherever your skills and interests lie. For autopilot tests specifically, check out this recipe on developer.ubuntu.com and watch this video.

I look forward to merging your work!

Testing all the things: Dropping Letters

It's almost FRIDAY! To celebrate, let's look at the second game that's become part of the core apps, dropping letters.

That best score! It was my first game, I can do much better now :)

The name of the game in dropping letters is to form words as the letters fall and try and prevent the screen from filling up. At the moment, the game is dogfoodable, but needs your help for tests. There are still some bugs within the code, and autopilot tests will help ensure those bugs don't sneak back into the codebase.

Consider helping the dropping letters developers keep development going.  Grab the dropping letters branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, check out the logs, and a FAQ from the workshops. Feel free to contact me with any questions or help. Happy test writing!

Testing all the things: File Manager

Today we look at another important piece of the puzzle for the ubuntu touch platform; the file manager. File managers provide a way for the end users to access there files in an arbitrary manner. Without one on the device you would be stuck viewing your data only through specific applications intended to utilize that data (photos, music, videos) or the terminal. Despite being the bane of simple computing advocates at times, file managers serve an important purpose.

So, let's look at the file manager app for ubuntu touch. Glancing at the dogfooding page you can see many of the basics are already in place. We can browse files and folders, make new folders and view file information.

Quite nice looking isn't it?

And indeed, looking at the list of needs many tests have already been written. This is largely the work of iBelieve, otherwise known as Michael Spencer. Good work! But there is still more to do.

If you are new to writing tests for core applications, adding a test to this application will be much easier for you to pick up. I'm sure Michael won't mind that I'm volunteering him here as well -- we're here to help!

Consider helping Michael and the file manager developers keep development going.  Grab the file manager branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, there is 1 more workshops planned for this week. In addition the logs, and a FAQ from the workshops are now available and ready for you to consume and understand. Happy test writing!

Testing all the things: Calculator

The calculator app is probably the biggest fashion saving application (for me!) on the platform. You might be scratching your heads, so let me share a quick story.

You see not unlike the crazy pebble watch idea, there was a time when I wore a crazy watch on my wrist.

Remember these?

I proudly wore my calculator watch for maybe a year before breaking the band after catching the giant display on something :-) That was the last time a watch graced my arm in a permanent fashion. Ahh the memories.

So, this app will make sure such a fashion disaster doesn't have to happen again. Thanks to the calculator app, I can still perform those geeky tasks of calculating out gas mileage, perfect tipping, supermarket costs, etc without the paying the ultimate fashion price.

The foundations are already in place for you to add tests, but more are needed, including testing the cool "tear-off" feature, swiping to delete, and historical calculations.

Consider helping the calculator developers keep development going.  Grab the calculator branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, there are 2 more workshops planned for this week. In addition the logs, and a FAQ from the workshops are now available and ready for you to consume and understand. Happy test writing!

Testing all the things: Stock Ticker

Ahh, Monday morning! It's time to look at the news headlines and check the stock market. As usual, your ubuntu phone has you covered for this part of your morning routine! Enter stock ticker!

As one of the newest core apps, Stock Ticker has done it's part in catching up quickly. The interface already sports working charting, news and detail views for stock symbols of your choosing via the management page.

 

Consider helping the stock ticker developers keep development going.  Grab the stock ticker branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, there are 2 more workshops planned for this week. In addition the logs, and a FAQ from the workshops are now available and ready for you to consume and understand. Happy test writing!

Dogfooding the core apps

So you've seen the building excitement and noise around the core apps project and are wishing there was a way for you to help. Perhaps your not a developer or someone with the skills to help write auomated tests. Or maybe you just want a preview of what things are like and play around with the developing ubuntu touch platform.

The good news is that you can! As Jono shared, we want to dogfood the core apps this month. The core apps can all be run on your ubuntu desktop, you don't need to flash a phone or tablet, and you don't even need to be running saucy (ubuntu development version). (Of course if you do have a phone, flash it and dogfood there if possible!)

Yummy!

To be fair, running and playing with the core apps is a lot more fun than eating actual dogfood, and it might even be healthier for you too :-p

Enter the core apps ppa. Just a quick command away and you can get access to all of the core apps. Ready?

EDIT: You also need the qt5 and ubuntu-sdk team ppa in order for the dependencies to work. Sorry!

sudo add-apt-repository ppa:canonical-qt5-edgers/qt5-proper
sudo add-apt-repository ppa:ubuntu-sdk-team/ppa
sudo apt-get update && sudo apt-get install ubuntu-sdk

then,

sudo add-apt-repository ppa:ubuntu-touch-coreapps-drivers/daily
sudo apt-get update && sudo apt-get install touch-coreapps

This will install the ppa and all of the core apps. They should run just fine on your desktop.

So now what? Well, even without a phone you can dogfood the apps a little. Try them out. Use them. Find bugs and problems. See if they meet your needs (real or percieved) for usage. Everyone's needs and usage will be different, and thus this early feedback is important to get into the hands of the developers.

Find a bug? See a potential missing feature? Check out the dogfooding page for bug reporting links and instructions as well as blueprints showing work items and status for features. Happy Testing err, Dogfooding!

Testing all the things: Weather

"What's the weather like? Got a window? Open it!"

Or I suppose I could swipe my gorgeous ubuntu phone and see my weather with lovely icons. I may just decide to stay inside and watch my virtual sun bask it's artificial light upon my skin for hours. I'm only (half) kidding.

Still it's not hard to admit the weather app is shaping up to be one of the sharpest looking applications for ubuntu touch. Martin Borho has already gotten through many of the bugs listed so there is an excellent structure in place for you to add some further tests.

Ok, so today isn't so lovely.

Consider helping the weather developers keep things looking sunny.  Grab the weather branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

If you need help getting started, there are 2 more workshops planned for next week. In addition the logs, and a FAQ from the workshops are now available and ready for you to consume and understand. Happy test writing!

Automated Testing Workshop: Day 1

A big thank you to everyone who made it to the first workshop for automated testing! For those of you who couldn't make it, let me remind you we have 3 more scheduled.

Friday July 5th at 1300 UTC
Tuesday July 9th at 1800 UTC
Thursday July 11th at 2200 UTC

But I also wanted to leave you with a quick log from the session introduction for you to peruse, as well as this list of questions from the session. I hope this helps those of you trying to learn the ropes and contribute new tests!

I'm including some of the questions below as a bit of an FAQ for those starting the journey. Read the workshop introduction and the FAQ below and go automate all the things! I'll see you at the next workshop.

What do I need to develop tests?
A raring or saucy installation (VM or real) and the autopilot and ubuntu-sdk packages installed. You will also need an understanding of how autopilot works (or be willing to learn :-) )

How do I run/install a core app?
Once you've branched your core app source code, you don't need to install it in order to run it. However there is a ppa with all the core apps you can install. To run the core app from source, run it like so in the root directory:
qmlscene APPNAME.qml, ie qmlscene dropping-letters.qml
In order to install from the ppa, follow the info here: https://wiki.ubuntu.com/Touch/CoreApps/PPA
You can install all the core apps run them as you would any other application. After installing from the ppa, simply run the application name, ie "dropping-letters"

I recieved an error installing from the ppa; qtdeclarative5-* missing, etc
You are missing the ubuntu-sdk and related packages. Install them using
sudo add-apt-repository ppa:canonical-qt5-edgers/qt5-proper && sudo add-apt-repository ppa:ubuntu-sdk-team/ppa && sudo apt-get update && sudo apt-get install ubuntu-sdk

Can we write tests for qml apps using autopilot using precise or quantal?
No, the ubuntu-sdk and the new version of autopilot we're utilizing both require raring and preferably saucy to work.

Where can I see an example of autopilot tests?
The tutorial on developer.ubuntu.com is an excellent first step for seeing an autopilot test in action and seeing an explanation of the test and how it works. In addition, the file manager, clock, weather, calendar and weather core apps already have some autopilot tests written as of this writing.

How much python does one need to know in order to write autopilot tests?
Not as much as you think :-) If you are familiar with programming and can understand and use the basic autopilot functions and the ubuntu sdk emulator, writing a test won't require you to learn any fancy python.

Testing all the things: Clock

When your like me and work with wonderful people across the world, knowing what time it is turns out to be really important. It's important to know what time it is in my timezone and the timezone of others I work with around the world.

Even sticking with my own timezone, I use my my clock to wake me each morning, keep track of my running, and let me time things while cooking (proper eggs anyone?) :-) Nekhelesh Ramananthan and the other clock developers are tackling all of these problems with the clock app. Sadly unlike sudoku, I haven't found a way how to cheat time.

Tick tock goes the clock

With that in mind, it's important the clock app gets it's fair share of testing! Nekhelesh has added some tests from the buglist, but some of the tests require further feature development. Since we can't stop time (well at least I can't!), it would be a great help to have someone come alongside these developers and add some testcases so they can focus on the application itself.

Consider helping the clock developers keep the clock regression free and well tested as it's features mature.  Grab the clock branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

Automated Testing Workshops

Recently we've been on a campaign to help increase the amount of automated tests we have for ubuntu. Specifically the effort is focused around helping out our community developers on the core apps project. The core apps project is building the core applications for ubuntu touch. Excellent stuff, all being done by the community!

The "testing all the things" blog series is currently covering each of these core applications and ends with a call to help the development teams. I've linked to tutorials like this and this on autopilot providing what you need to know. But sometimes seeing is understanding, and a helping hand can go a long way.

With that in mind, I am announcing a series of workshops to help you gather the skills needed to write automated tests. You can help contribute with just your ubuntu pc, writing and running tests without needing phone hardware! We're going to focus on autopilot, and for the moment the ubuntu core apps. I'll try and alternate to host them at timezone friendly times for everyone (granted I do have to sleep at some point too!). Here's the schedule, with links to the event on G+ page.

Tomorrow!, Wednesday July 3rd at 1800 UTC
Friday July 5th at 1300 UTC
Tuesday July 9th at 1800 UTC
Thursday July 11th at 2200 UTC

The workshops will take place in #ubuntu-quality and will all last an hour (but I won't leave you hanging if we need more time!). I'll host g+ hangouts and provide one on one help as needed to anyone writing tests. See you at the workshops!

Testing all the things: RSS Reader Shorts

In honor of the closing of google reader, I thought I would highlight another core application that needs some attention; namely the RSS Reader, with a proper name Shorts. If your already bored and yawning (RSS is dead, long live RSS), have a look at the design's recently shared by the design team as well as the original post with the user stories. Seems like RSS might not be so dead (or look it!)!

Yes, I still use RSS feeds, mainly as a news aggregator. In many ways honestly RSS feeds have long replaced my idea of bookmarking things. Bookmarks are general stale old content that never updates, is never refreshed and is eventually just purged. The ideas shown in the design of Shorts are great and the development team has a wonderful task ahead of them of implementing them.

With the development team focused on getting the code written, it's our opportunity to help out by adding testcases for there work. For instance, simple things like adding, editing, and removing a rss feed all need tested. The testcases are ready and waiting for you to add a test!

Consider helping the shorts developers get everything in shape. Grab the rss reader branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

Testing all the things: Sudoku

Coming off a lovely weekend, it's time we turned our attention to an app on the lighter side. Anyone up for a game of sudoku?

Sudoku is an example of a simple logic game that can be learned easily enough yet has the staying power to intrigue me to continue to play it. Dinko Osmankovic and the rest of the Sudoku Touch developers have created a version for ubuntu touch to fill those critical mundane moments of the day -- waiting for a train or having your morning coffee. Or perhaps if your like me, fighting insomnia (yikes!).

Apparently using the show hints button to play the entire game makes me a cheat.

So, while it seems the game is smart enough to slap me for trying to cheat my way through, it needs some testcases! Looking at the buglist, there are seven tasty bugs with your potential name on them. This is testing at it's finest! It's rare to count playing a game as helping ubuntu -- but in this case, you would be right!

Themes support!

Consider helping the sudoku touch developers as the game and it's features mature.  Grab the sudoku branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping to ensure quality for ubuntu touch!

Mir joins Cadence Testing

With the announcement of MIR being the default display server for 13.10, many folks rightly wonder if it will be stable and ready to ship by then. Well, as part of ensuring that will be the case, I'd like to announce that XMir will become part of the bi-weekly cadence testing we as a community team undertake. Week 2 begins this Saturday and will include XMir testing.

The testcases we'll be using at the moment are basic smoke tests to check the overall state of XMir. As the cycle wears on our goal is to perform a full regression test for XMir against the normal Xorg server to make sure everything is super smooth. The goal is for someone running ubuntu saucy to not even realize something has changed with the display server. So ready to help?

Check out this page to learn about how to install Mir. Those same instructions are linked from the testcase itself. Run through the tests listed and report your results. Make sure your logged in or you won't be able to report anything :-)

New to cadence testing or the tracker? Here's some links you might find useful:

Understanding how to use the QATracker
Understanding how to perform a cadence test

There's video versions too!

QATracker
Cadence Testing
 
Thanks for helping test ubuntu! Your willingness to live on the edge and test ensure others of proper functioning software in the stable release.

Testing all the things: Calendar

A good calendar is essential to me. I'm liable to forget almost everything about my day except eating :-) Things like day of the week and month are important details I definitely rely on a calendar for (I can usually get the month right!).

Fortunately for me (and you!), there is a core app that provides a handy Calendar. Michael Hall featured this application a few weeks ago on his blog covering a development rundown of the application. It covers the list of features nicely. In a word, there's a lot of neat stuff to test in there.

Looking at the buglist of needs there are only 2 showing in-progress -- plenty of room for someone to help out by testing each one of the different views. Monthly, daily, weekly; accessed via swiping.

" Swiping left and right on the month will take you back or forward a month at a time.  Swiping left or right on the bottom half will take you back and forward a day at a time.

Pull the event area down and let it go, and the month will collapse down into a single week. Now swiping left and right there will move you back and forward a week at a time.  Pull down and let it go again and it will snap back to showing the full month.

Finally, you have an option in the toolbar (swipe up from the bottom edge) to switch from an event list to a timeline view of your events."

Are you dizzy yet?

These seamless transitions could use some cool testcases! At the moment, the app is seeing it's first merge requests being made by Carla Sella and Kunal Parmar. The team has faced some issues with uncovering some unique requirements for autopilot, which have now been fixed. Excellent work both of you!

Consider helping Carla, Kunal and the ubuntu calendar developers as the application and it's features mature.  Grab the calendar branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping be a part of ubuntu!

Testing all the things: Terminal App

I couldn't help but start with one of the core apps I consider essential (to me anyway!) on my phone, a terminal. The terminal app being developed for ubuntu has some wonderful features built with a touch interface in mind. One of the biggest issues with touch is having a terminal ready keyboard with things like page up and down, arrow keys, and not to mention being able to use keyboard shortcuts like ctrl+d, ctrl+z, ctrl+c, etc. This has been handled rather elegantly with a long tap menu as you can see below, in addition to a panel that optionally appears at the top of the application.

Dmitry Zagnoyko has already landed a few tests for some of the features present, as you can see below. Execellent work Dmitry! A basic testcase exists now for each of the panels and the circle menu.

Help Dmitry and the terminal app team make sure all the features work properly for you upon release. Get involved and add a test. The initial setup work has already been done, and there are existing testcases already written. Grab the terminal branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge.