With the announcement of MIR being the default display server for 13.10, many folks rightly wonder if it will be stable and ready to ship by then. Well, as part of ensuring that will be the case, I'd like to announce that XMir will become part of the bi-weekly cadence testing we as a community team undertake. Week 2 begins this Saturday and will include XMir testing.
The testcases we'll be using at the moment are basic smoke tests to check the overall state of XMir. As the cycle wears on our goal is to perform a full regression test for XMir against the normal Xorg server to make sure everything is super smooth. The goal is for someone running ubuntu saucy to not even realize something has changed with the display server. So ready to help?
Check out this page to learn about how to install Mir. Those same instructions are linked from the testcase itself. Run through the tests listed and report your results. Make sure your logged in or you won't be able to report anything :-)
New to cadence testing or the tracker? Here's some links you might find useful:
Understanding how to use the QATracker
Understanding how to perform a cadence test
There's video versions too!
QATracker
Cadence Testing
Thanks for helping test ubuntu! Your willingness to live on the edge and test ensure others of proper functioning software in the stable release.
Friday, June 28, 2013
Testing all the things: Calendar
A good calendar is essential to me. I'm liable to forget almost everything about my day except eating :-) Things like day of the week and month are important details I definitely rely on a calendar for (I can usually get the month right!).
Fortunately for me (and you!), there is a core app that provides a handy Calendar. Michael Hall featured this application a few weeks ago on his blog covering a development rundown of the application. It covers the list of features nicely. In a word, there's a lot of neat stuff to test in there.
Looking at the buglist of needs there are only 2 showing in-progress -- plenty of room for someone to help out by testing each one of the different views. Monthly, daily, weekly; accessed via swiping.
" Swiping left and right on the month will take you back or forward a month at a time. Swiping left or right on the bottom half will take you back and forward a day at a time.
Fortunately for me (and you!), there is a core app that provides a handy Calendar. Michael Hall featured this application a few weeks ago on his blog covering a development rundown of the application. It covers the list of features nicely. In a word, there's a lot of neat stuff to test in there.
Looking at the buglist of needs there are only 2 showing in-progress -- plenty of room for someone to help out by testing each one of the different views. Monthly, daily, weekly; accessed via swiping.
" Swiping left and right on the month will take you back or forward a month at a time. Swiping left or right on the bottom half will take you back and forward a day at a time.
Pull the event area down and let it go, and the
month will collapse down into a single week. Now swiping left and right
there will move you back and forward a week at a time. Pull down and
let it go again and it will snap back to showing the full month.
Finally, you have an option in the toolbar
(swipe up from the bottom edge) to switch from an event list to a
timeline view of your events."
Are you dizzy yet?
These seamless transitions could use some cool testcases! At the moment, the app is seeing it's first merge requests being made by Carla Sella and Kunal Parmar. The team has faced some issues with uncovering some unique requirements for autopilot, which have now been fixed. Excellent work both of you!
Consider helping Carla, Kunal and the ubuntu calendar developers as the application and it's features mature. Grab the calendar branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge. Thanks for helping be a part of ubuntu!
Labels:
autopilot,
opportunity,
quality,
testing,
ubuntu
Thursday, June 27, 2013
Testing all the things: Terminal App
I couldn't help but start with one of the core apps I consider essential (to me anyway!) on my phone, a terminal. The terminal app being developed for ubuntu has some wonderful features built with a touch interface in mind. One of the biggest issues with touch is having a terminal ready keyboard with things like page up and down, arrow keys, and not to mention being able to use keyboard shortcuts like ctrl+d, ctrl+z, ctrl+c, etc. This has been handled rather elegantly with a long tap menu as you can see below, in addition to a panel that optionally appears at the top of the application.
Dmitry Zagnoyko has already landed a few tests for some of the features present, as you can see below. Execellent work Dmitry! A basic testcase exists now for each of the panels and the circle menu.
Help Dmitry and the terminal app team make sure all the features work properly for you upon release. Get involved and add a test. The initial setup work has already been done, and there are existing testcases already written. Grab the terminal branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge.
Dmitry Zagnoyko has already landed a few tests for some of the features present, as you can see below. Execellent work Dmitry! A basic testcase exists now for each of the panels and the circle menu.
Help Dmitry and the terminal app team make sure all the features work properly for you upon release. Get involved and add a test. The initial setup work has already been done, and there are existing testcases already written. Grab the terminal branch, add a testcase from the list of needs, follow the tutorial for help if needed, and propose a merge.
Labels:
autopilot,
opportunity,
quality,
testing,
ubuntu
Testing all the things: Introduction
As a quality community team, we've been continuing to make progress this cycle on automating our testcases, especially the new applications that are being written for ubuntu touch. These 'core apps' are being written by other community members for the next generation of ubuntu.
We're also making progress on our desktop applications and automating the ubiquity installer. With that in mind, I'm going to start a little blog series highlighting a package a day for automating. I'll dub it rather unoriginally "Testing All The Things". My goal is to showcase the wonderful work going on with testing this cycle in ubuntu, but also to encourage you dear reader to get involved in helping us. All areas of ubuntu (flavors too!) can benefit from some robot friends helping test the packages they work on and utilize.
But you don't need to wait to see your favorite app hit the list. Hit up the tutorials below for information to dive in and help us!
Core Apps Test Wiki
Writing an autopilot test for ubuntu sdk applications
QML Autopilot Tutorial with example application
Autopilot Tests Project
Writing an autopilot test for desktop applications
We're also making progress on our desktop applications and automating the ubiquity installer. With that in mind, I'm going to start a little blog series highlighting a package a day for automating. I'll dub it rather unoriginally "Testing All The Things". My goal is to showcase the wonderful work going on with testing this cycle in ubuntu, but also to encourage you dear reader to get involved in helping us. All areas of ubuntu (flavors too!) can benefit from some robot friends helping test the packages they work on and utilize.
But you don't need to wait to see your favorite app hit the list. Hit up the tutorials below for information to dive in and help us!
Core Apps Test Wiki
Writing an autopilot test for ubuntu sdk applications
QML Autopilot Tutorial with example application
Autopilot Tests Project
Writing an autopilot test for desktop applications
Labels:
autopilot,
opportunity,
quality,
testing,
ubuntu
Friday, June 21, 2013
Update on automated testing with autopilot
Given all recent love and excitement for autopilot I wanted to share the QA community's progress on writing autopilot tests for, celebrate our successes and let everyone know where we still need help.
First let's talk about the ubuntu-autopilot-tests project. As part of the hackfests held at the end of May/early June we were able to complete the transition to autopilot 1.3 of the ubuntu desktop autopilot tests. Thanks to all of the contributors and hackers for helping on this! In addition, we now have a production branch, and the canonical platform QA team is working on adding them to the official smoke testing each day, Great work everyone! That said, tests are still needed, and in some cases the testcases are still basic and not covering many of the application features. There is still room for you to be invovled! Of note is the on-going work to automated our image testing via the UI.
Next, let's talk about the core apps. Last Thursday we held a hackfest to help kickstart testcases for all of these projects. So let's take a look at how far we've come in a week. As a reminder, testcase contributions to any of the core apps is very much appreciated -- there is still a need for you to come alongside and help write tests!
Calculator
There are already several testcases merged in with the main branch, but as one of the most feature complete applications, work and help is still needed in this area. There are currently 6 open bugs for tests needed here. This is a great application to contribute to for someone new to autopilot!
Calendar
There are two pending merge requests and the work is underway towards knocking out the rest of the testcases needed.
Clock/Alarm
The clock team has jumped in headfirst to help with testcases. You can view the status of the remaining tests needed here.
Doc Viewer
I started on a branch for this and the basic infrastructure is in place. Branch the application. Grab a copy of the emulator, pick a bug and write your test. This app needs you!
File Manager
The first merge and test is in review. But there's still more tests to be written. Have a look at the list of needed tests.
Terminal
The first merge request has just been approved and landed for terminal autopilot tests. But there's more features to be tested in this awesome app. Grab something off the list and go. The setup work is already done.
Music
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!
Weather
Half of the initial testcases have been started and the first merges are being proposed. Rock on Martin!
Remember you can always view the big master list of all the open tests here. We've got a bit of work ahead of us! Be a part of the team. Grab an open bug from the list above or contact me for help and I'll make sure you get invovled!
First let's talk about the ubuntu-autopilot-tests project. As part of the hackfests held at the end of May/early June we were able to complete the transition to autopilot 1.3 of the ubuntu desktop autopilot tests. Thanks to all of the contributors and hackers for helping on this! In addition, we now have a production branch, and the canonical platform QA team is working on adding them to the official smoke testing each day, Great work everyone! That said, tests are still needed, and in some cases the testcases are still basic and not covering many of the application features. There is still room for you to be invovled! Of note is the on-going work to automated our image testing via the UI.
Next, let's talk about the core apps. Last Thursday we held a hackfest to help kickstart testcases for all of these projects. So let's take a look at how far we've come in a week. As a reminder, testcase contributions to any of the core apps is very much appreciated -- there is still a need for you to come alongside and help write tests!
Calculator
There are already several testcases merged in with the main branch, but as one of the most feature complete applications, work and help is still needed in this area. There are currently 6 open bugs for tests needed here. This is a great application to contribute to for someone new to autopilot!
Calendar
There are two pending merge requests and the work is underway towards knocking out the rest of the testcases needed.
Clock/Alarm
The clock team has jumped in headfirst to help with testcases. You can view the status of the remaining tests needed here.
Doc Viewer
I started on a branch for this and the basic infrastructure is in place. Branch the application. Grab a copy of the emulator, pick a bug and write your test. This app needs you!
File Manager
The first merge and test is in review. But there's still more tests to be written. Have a look at the list of needed tests.
RSS Reader
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!
The first merge request has just been approved and landed for terminal autopilot tests. But there's more features to be tested in this awesome app. Grab something off the list and go. The setup work is already done.
Music
Ready and waiting! Check out the list of bugs and have at it! The basic structure is already in place. Simply grab a copy of the emulator, pick a bug and write your test. This app needs you!
Weather
Half of the initial testcases have been started and the first merges are being proposed. Rock on Martin!
Remember you can always view the big master list of all the open tests here. We've got a bit of work ahead of us! Be a part of the team. Grab an open bug from the list above or contact me for help and I'll make sure you get invovled!
Thursday, June 20, 2013
An autopilot emulator for ubuntu sdk apps
A couple weeks ago we announced the initiative to drive up our autopilot (that is, automated) tests for our ubuntu touch core apps. The core apps are being made with the ubuntu sdk, and thus share the same language (QML) and toolkit (ubuntusdk).
With this in mind I wanted to provide an emulator, which in autopilot speak, is a utility class for writing autopilot tests that use the ubuntu SDK. The goal is to help accelerate the process for getting the testcases written, as well as standardizing best practices for testing common features. At the moment the emulator contains useful functions like tab switching, selecting from popovers, opening and closing the toolbar and clicking toolbar buttons. Please, take a look and utilize the emulator when you are contributing new tests for the ubuntu touch applications. For the moment, the emulator can be found here:
lp:~nskaggs/
The future home is hopefully in the SDK itself, but for now consider that branch your source for emulator goodness. Now, a quick FAQ.
Is it ready for use?
Yes, it's ready and tested on several core apps now including clock, calendar, terminal, and file manager. That said if you find an issue, simply contact me or propose an improvement!
How do I use it?
Inside your autopilot test subfolder, add an emulators folder if it's not already present. Next, branch my source above -- it will add ubuntusdk.py to the folder. Simply incorporate it into your __init__.py or testcase itself and call the utility functions with ubuntusdk.*. For an example check out the ubuntu-terminal-app and the merge request from today. It shows adding autopilot tests to an empty branch. In addition, the emulator (albeit an earlier version) was used in the tutorial on the ubuntu app developer portal.
Will it be updated?
Yes! Expect refinements and tweaks as we go along. Hopefully a true "stable and complete" version will appear in the not too distant feature when the emulator itself has a proper home. In the meantime, use it and as more complex tests are added, expect to update the emulator in the source branch you are working in.
Go forth and write tests!
With this in mind I wanted to provide an emulator, which in autopilot speak, is a utility class for writing autopilot tests that use the ubuntu SDK. The goal is to help accelerate the process for getting the testcases written, as well as standardizing best practices for testing common features. At the moment the emulator contains useful functions like tab switching, selecting from popovers, opening and closing the toolbar and clicking toolbar buttons. Please, take a look and utilize the emulator when you are contributing new tests for the ubuntu touch applications. For the moment, the emulator can be found here:
lp:~nskaggs/
The future home is hopefully in the SDK itself, but for now consider that branch your source for emulator goodness. Now, a quick FAQ.
Is it ready for use?
Yes, it's ready and tested on several core apps now including clock, calendar, terminal, and file manager. That said if you find an issue, simply contact me or propose an improvement!
How do I use it?
Inside your autopilot test subfolder, add an emulators folder if it's not already present. Next, branch my source above -- it will add ubuntusdk.py to the folder. Simply incorporate it into your __init__.py or testcase itself and call the utility functions with ubuntusdk.*. For an example check out the ubuntu-terminal-app and the merge request from today. It shows adding autopilot tests to an empty branch. In addition, the emulator (albeit an earlier version) was used in the tutorial on the ubuntu app developer portal.
Will it be updated?
Yes! Expect refinements and tweaks as we go along. Hopefully a true "stable and complete" version will appear in the not too distant feature when the emulator itself has a proper home. In the meantime, use it and as more complex tests are added, expect to update the emulator in the source branch you are working in.
Go forth and write tests!
Wednesday, June 19, 2013
QATracker Survey + bonus mockup
Hot on the heels of our first cadence week, I wanted to take the opportunity to collect feedback about the tools we as a community utilize. Specifically the QATracker which we heavily rely on for managing our work, testcases and results. From the wiki, "The
QATracker is the master repository for all our our testing within
ubuntu QA. It holds our testcases, records our results, and helps
coordinate our testing events."
This is a link to a brief survey asking a few simple questions about how you've used the tool. All your responses are anonymous, but I will publish the aggregate question information and share it with the community once completed. The goal is to help ensure the tool is meeting our needs and is being utilized.
I'll leave the survey up until June 24th. My hope is to encourage more folks to help test as well as make it more enjoyable for those already taking part. I want to ensure our tools and processes continue to evolve, strengthen and become more robust for everyone as we continue on our mission. Part of that is making sure the tools we use are enjoyable!
Thanks in advance everyone!
As a bonus, Pasi, aka knome, has put together some mockups on how we might be able to switch what the results page looks like. This is perhaps the most utilized page of the site, so without further ado, here's a mockup of some changes proposed to make it more usable:
Old Site
New Site Mockup
What a change eh? The add test results has been moved to the sidebar and simplified, the bugs listing has been written out, and the results have been moved to the top. Finally the links have also been moved to the sidebar and Pasi has updated the icons ;-)
SO, what does everyone think about the changes? Many thanks to Pasi for putting this together! Leave a comment, a message on the mailing list, or reflect your thoughts in the survey.
This is a link to a brief survey asking a few simple questions about how you've used the tool. All your responses are anonymous, but I will publish the aggregate question information and share it with the community once completed. The goal is to help ensure the tool is meeting our needs and is being utilized.
I'll leave the survey up until June 24th. My hope is to encourage more folks to help test as well as make it more enjoyable for those already taking part. I want to ensure our tools and processes continue to evolve, strengthen and become more robust for everyone as we continue on our mission. Part of that is making sure the tools we use are enjoyable!
Thanks in advance everyone!
As a bonus, Pasi, aka knome, has put together some mockups on how we might be able to switch what the results page looks like. This is perhaps the most utilized page of the site, so without further ado, here's a mockup of some changes proposed to make it more usable:
Old Site
New Site Mockup
What a change eh? The add test results has been moved to the sidebar and simplified, the bugs listing has been written out, and the results have been moved to the top. Finally the links have also been moved to the sidebar and Pasi has updated the icons ;-)
SO, what does everyone think about the changes? Many thanks to Pasi for putting this together! Leave a comment, a message on the mailing list, or reflect your thoughts in the survey.
Monday, June 17, 2013
Feeling Saucy? Help us Test the new release!
Join the ubuntu quality
community team's effort this week! As a community we test different
things about every ~2 weeks in ubuntu, and share the results to flesh
out bugs and problem areas.
So what's up for testing this week? The daily images, the default applications in ubuntu and a new version of the sound stack for testing.
Ready to help? Full details are here.
Need some help on how to contribute? Have a look at this page and the walkthroughs listed. Of particular interest is the ISO testing and Cadence Week testing walkthroughs.
Do note that you don't need anything special to participate in cadence week testing! Both an installed version of the development branch of ubuntu (aka saucy) in a VM or on a real box, or even a live session of the latest daily image will work. For more information on how to use a live session to test, check out the Cadence Week testing walkthrough or watch the youtube video of the same.
So what's up for testing this week? The daily images, the default applications in ubuntu and a new version of the sound stack for testing.
Ready to help? Full details are here.
Need some help on how to contribute? Have a look at this page and the walkthroughs listed. Of particular interest is the ISO testing and Cadence Week testing walkthroughs.
Do note that you don't need anything special to participate in cadence week testing! Both an installed version of the development branch of ubuntu (aka saucy) in a VM or on a real box, or even a live session of the latest daily image will work. For more information on how to use a live session to test, check out the Cadence Week testing walkthrough or watch the youtube video of the same.
Happy Testing!
Wednesday, June 5, 2013
Autopilot Tests for Ubuntu Touch Core Apps
A few months ago the ubuntu touch core apps project was launched. For those of you following along with Michael's regular updates have gotten to see these applications grow up rather quickly.
Autopilot Says: How can I help?
Now it's time to add some more testing around these applications as they have reached a basic functional level of usability. Automated testing via autopilot to the rescue!
To help kickstart this process we've put together a recipe for writing autopilot tests specific to QML applications and added it to developer.ubuntu.com. In addition, we'll be hosting a hackfest next week on June 13th to help add basic autopilot testcases for each of the core apps. Folks will be on-hand ready to field your questions and hack together on the autopilot testcases needed for the applications. Join us and help support the wonderful community of application developers making awesome applications for ubuntu!
So how can you help?
Now it's time to add some more testing around these applications as they have reached a basic functional level of usability. Automated testing via autopilot to the rescue!
To help kickstart this process we've put together a recipe for writing autopilot tests specific to QML applications and added it to developer.ubuntu.com. In addition, we'll be hosting a hackfest next week on June 13th to help add basic autopilot testcases for each of the core apps. Folks will be on-hand ready to field your questions and hack together on the autopilot testcases needed for the applications. Join us and help support the wonderful community of application developers making awesome applications for ubuntu!
So how can you help?
- First, go read through the recipe on writing autopilot tests for QML applications. It's also a good idea to have a look through the official tutorial for autopilot and bookmark the API reference link so it's handy.
- Armed with your new knowledge, start hacking on some autopilot tests for the core apps. Here's a list of core applications along with the status of autopilot tests. Choose something that looks interesting to you and add some tests.
- Follow the contributing guide to help you get your work contributed into the ubuntu touch core application project you chose.
- Finally come out to the hackfest! It's your chance to share your work, ask questions, get your tests sorted and merged and socialize and meet other members of the community.
- Don't forget there is a wonderful quality community you can be a part of and get help from if you get stuck! There's a mailing list for ubuntu-touch, and ubuntu-quality as well as IRC channels #ubuntu-touch, #ubuntu-autopilot and #ubuntu-quality. Use these resources to help you!
Labels:
opportunity,
quality,
testing,
ubuntu
Subscribe to:
Posts (Atom)