I thought I would add a little festivity to the holiday season, quality style. In case your holidays just are not the same without a little quality in your life, allow me to share how you can get involved.
There are opportunities for every role listed on the QA wiki. Testers and test writers are both needed. Testing and writing manual tests can be learned
by anyone, no coding required. That said if you have skills or interest
in technical work, I would encourage you help out. You will learn by doing and get help from others while you do it.
Now onto the good stuff! What can you do to help ubuntu this cycle from a quality perspective?
Dogfooding
There is an ever present need for brave folks willing to simply run the development version of ubuntu and use it as a daily machine throughout the cycle. It's one of the best ways for us as a community to uncover bugs and issues, in particular things that regress from the previous release. Upgrade to vivid today and see what you can break!
QATracker
This tool is written in drupal7 and runs the iso.qa.ubuntu.com and
packages.qa.ubuntu.com sites. These sites are used to record and view
the results of all of our manual testing efforts. Currently dkessel is
leading the effort on implementing some needed UI changes. The code and more information
about the project can be found on launchpad. The tracker is one of our primary tools and needs your help to become friendly for everyone to use.
In addition a charm would be useful to simplify setting up a
development environment. The charm can be based upon the existing drupal charm. At the moment this work is
ready for someone to jump in.
Unity8
Running unity8 as a full-time desktop is a personal goal I have for this
cycle. I hope some others might also want to be early adopters and join me in this goal. For now you can help by testing the unity8 desktop. Have a
look at running unity in lxc for an easy way to run
unity8 today on your machine. Use it, test it, and offer
feedback. I'll be talking more about unity8 as the cycle progresses and
opportunities to test new features aimed at the desktop appear.
Core Apps
The core apps project is an excellent way to get involved. These applications have been lovingly developed by community members just like you. Many of the teams are looking for help in writing tests and for someone who can help bring a testing mindset and eye to the work. As of this writing specifically the docviewer, terminal and calculator teams would love your help. The core apps hackdays are happening this week, drop by and introduce yourself to get started!
Manual Tests
Like the sound of writing tests but the idea of writing code turns you off? Manual tests are needed as well! They are written in English and are easy to understand and write. Manual tests include everything you see on the qatracker and are managed as a launchpad project. This means you can pick a bug and "fix it" by submitting a merge request. The bugs involve both fixing existing tests as well as requests for new testcases.
Images
As always there are images that need testing. Testing milestones occur later in the cycle which involve everyone helping to test a specific set of images. In the meantime, daily images are generated that have made it through the automated tests and are ready for manual testing. Booting an image in a live session is a great way to check for regressions on your machine. Doing this early in the cycle can help make sure your hardware and others like it experience a regression free upgrade when the time comes.
Triaging
After subjecting software to testing, bugs are naturally found. These bugs then need to be verified and triaged. The bugsquadders, as they are called, would be happy to help you learn to categorize or triage bugs and do other tasks.
No matter how you choose to get involved, feel free to contact me for help if needed. Most of all, Happy Testing!
Tuesday, December 9, 2014
Tuesday, December 2, 2014
Creating multi-arch click packages
Click packages are one of the pieces of new technology that drives the next version of ubuntu on the phone and desktop. In a nutshell click packages allow for application developers to easily package and deliver application updates independent of the distribution release or archive. Without going into the interesting technical merits and de-merits of click packages, this means the consumer can get faster application updates. But much of the discussion and usage of click packages until now has revolved around mobile. I wanted to talk about using click packages on the desktop and packaging clicks for multiple architectures.
The manifest file
Click packages follow a specific format. Click packages contain a payload of an application's libraries, code, artwork and resources, along with its needed external dependencies. The description of the package is found in the manifest file, which is what I'd like to talk about. The file must contain a few keys, but one of the recognized optional keys is architecture. This key allows specifying architectures the package will run on.
If an application contains no compiled code, simply use 'all' as the value for architecture. This accomplishes the goal of running on all supported architectures and many of the applications currently in the ubuntu touch store fall into this category. However, an increasing number of applications do contain compiled code. Here's how to enable support across architectures for projects with compiled code.
Fat packages
The click format along with the ubuntu touch store fully support specifying one or more values for specific architecture support inside the application manifest file. Those values follow the same format as dpkg architecture names. Now in theory if a project containing compiled code lists the architectures to support, click build should be able to build one package for all. However, for now this process requires a little manual intervention. So lets talk about building a fat (or big boned!) package that contains support for multiple architectures inside a single click package.
Those who just want to skip ahead can check out the example package I put together using clock. This same package can be found in the store as multi-arch clock test. Feel free to install the click package on the desktop, the i386 emulator and an armhf device.
Building a click for a different architecture
To make a multi-arch package a click package needs to be built for each desired architecture. Follow this tutorial on developer.ubuntu.com for more information on how to create a click target for each architecture. Once all the targets are setup, use the ubuntu sdk to build a click for each target. The end result is a click file specific to each architecture.
For example in creating the clock package above, I built a click for amd64, i386 and armhf. Three files were generated:
com.ubuntu.clock_3.2.176_amd64.click
com.ubuntu.clock_3.2.176_i386.click
com.ubuntu.clock_3.2.176_armhf.click
Notice the handy naming scheme allows for easy differentiation as to which click belongs to which architecture. Next, extract the compiled code from each click package. This can be accomplished by utilizing dpkg. For example,
dpkg -x com.ubuntu.clock_3.2.176_amd64.click amd64
Do this for each package. The result should be a folder corresponding to each package architecture.
Next copy one version of the package for use as the base of multi-arch click package. In addition, remove all the compiled code under the lib folder. This folder will be populated with the extracted compiled code from the architecture specific click packages.
cp amd64 multi
rm -rf multi/lib/*
Now there is a folder for each click package, and a new folder named multi that contains the application, minus any compiled code.
Creating the multi-arch click
Inside the extracted click packages is a lib folder. The compiled modules should be arranged inside, potentially inside an architecture subfolder (depending on how the package is built).
Copy all of the compiled modules into a new folder inside the lib folder of the multi directory. The folder name should correspond to the architecture of the complied code. Here's a list of the architectures for ARM, i386, and amd64 respectively.
arm-linux-gnueabihf
i386-linux-gnu
x86_64-linux-gnu
You can check the naming from an intended device by looking in the application-click.conf file.
grep ARCH /usr/share/upstart/sessions/application-click.conf
To use the clock package as an example again, here's a quick look at the folder structure:
lib/arm-linux-gnueabihf/...
lib/i386-linux-gnu/...
lib/x86_64-linux-gnu/...
The contents of lib/* from each click package I built earlier is under a corresponding folder inside the multi/lib directory. So for example, the lib folder from com.ubuntu.clock_3.2.176_i386.click became lib/i386-linux-gnu/.
Presto, magic package time!
Finally the manifest.json file needs to be updated to reflect support for the desired architectures. Inside the manifest.json file under the multi directory, edit the architecture key values to list all supported architectures for the new package. For example to list support for ARM and x86 architectures,
"architecture": ["armhf", "i386", "amd64"],
To build the new package, execute click build multi. The resulting click should build and be named with a _multi.click prefix. This click can be installed on any of the specified architectures and is ready to be uploaded to the store.
Caveats, nibbly bits and bugs
So apart from click not automagically building these packages, there is one other bug as of this writing. The resulting multi-arch click will fail the automated store review and instead enter manual review. To workaround this request a manual review. Upon approval, the application will enter the store as usual.
Summary
In summary to create a multi-arch click package build a click for each supported architecture. Then pull the compiled library code from each click and place into a single click package. Next modify the click manifest file to state all of the architectures supported. Finally, rebuild the click package!
I trust this explanation and example provides encouragement to include support for x86 platforms when creating and uploading a click package to the store. Undoubtedly there are other ways to build a multi-arch click; simply ensure all the compiled code for each architecture is included inside the click package. Feel free to experiment!
If you have any questions as usual feel free to contact me. I look forward to seeing more applications in the store from my unity8 desktop!
The manifest file
Click packages follow a specific format. Click packages contain a payload of an application's libraries, code, artwork and resources, along with its needed external dependencies. The description of the package is found in the manifest file, which is what I'd like to talk about. The file must contain a few keys, but one of the recognized optional keys is architecture. This key allows specifying architectures the package will run on.
If an application contains no compiled code, simply use 'all' as the value for architecture. This accomplishes the goal of running on all supported architectures and many of the applications currently in the ubuntu touch store fall into this category. However, an increasing number of applications do contain compiled code. Here's how to enable support across architectures for projects with compiled code.
Fat packages
The click format along with the ubuntu touch store fully support specifying one or more values for specific architecture support inside the application manifest file. Those values follow the same format as dpkg architecture names. Now in theory if a project containing compiled code lists the architectures to support, click build should be able to build one package for all. However, for now this process requires a little manual intervention. So lets talk about building a fat (or big boned!) package that contains support for multiple architectures inside a single click package.
Those who just want to skip ahead can check out the example package I put together using clock. This same package can be found in the store as multi-arch clock test. Feel free to install the click package on the desktop, the i386 emulator and an armhf device.
Building a click for a different architecture
To make a multi-arch package a click package needs to be built for each desired architecture. Follow this tutorial on developer.ubuntu.com for more information on how to create a click target for each architecture. Once all the targets are setup, use the ubuntu sdk to build a click for each target. The end result is a click file specific to each architecture.
For example in creating the clock package above, I built a click for amd64, i386 and armhf. Three files were generated:
com.ubuntu.clock_3.2.176_amd64.click
com.ubuntu.clock_3.2.176_i386.click
com.ubuntu.clock_3.2.176_armhf.click
Notice the handy naming scheme allows for easy differentiation as to which click belongs to which architecture. Next, extract the compiled code from each click package. This can be accomplished by utilizing dpkg. For example,
dpkg -x com.ubuntu.clock_3.2.176_amd64.click amd64
Do this for each package. The result should be a folder corresponding to each package architecture.
Next copy one version of the package for use as the base of multi-arch click package. In addition, remove all the compiled code under the lib folder. This folder will be populated with the extracted compiled code from the architecture specific click packages.
cp amd64 multi
rm -rf multi/lib/*
Now there is a folder for each click package, and a new folder named multi that contains the application, minus any compiled code.
Creating the multi-arch click
Inside the extracted click packages is a lib folder. The compiled modules should be arranged inside, potentially inside an architecture subfolder (depending on how the package is built).
Copy all of the compiled modules into a new folder inside the lib folder of the multi directory. The folder name should correspond to the architecture of the complied code. Here's a list of the architectures for ARM, i386, and amd64 respectively.
arm-linux-gnueabihf
i386-linux-gnu
x86_64-linux-gnu
You can check the naming from an intended device by looking in the application-click.conf file.
grep ARCH /usr/share/upstart/sessions/application-click.conf
To use the clock package as an example again, here's a quick look at the folder structure:
lib/arm-linux-gnueabihf/...
lib/i386-linux-gnu/...
lib/x86_64-linux-gnu/...
The contents of lib/* from each click package I built earlier is under a corresponding folder inside the multi/lib directory. So for example, the lib folder from com.ubuntu.clock_3.2.176_i386.click became lib/i386-linux-gnu/.
Presto, magic package time!
Finally the manifest.json file needs to be updated to reflect support for the desired architectures. Inside the manifest.json file under the multi directory, edit the architecture key values to list all supported architectures for the new package. For example to list support for ARM and x86 architectures,
"architecture": ["armhf", "i386", "amd64"],
To build the new package, execute click build multi. The resulting click should build and be named with a _multi.click prefix. This click can be installed on any of the specified architectures and is ready to be uploaded to the store.
Caveats, nibbly bits and bugs
So apart from click not automagically building these packages, there is one other bug as of this writing. The resulting multi-arch click will fail the automated store review and instead enter manual review. To workaround this request a manual review. Upon approval, the application will enter the store as usual.
Summary
In summary to create a multi-arch click package build a click for each supported architecture. Then pull the compiled library code from each click and place into a single click package. Next modify the click manifest file to state all of the architectures supported. Finally, rebuild the click package!
I trust this explanation and example provides encouragement to include support for x86 platforms when creating and uploading a click package to the store. Undoubtedly there are other ways to build a multi-arch click; simply ensure all the compiled code for each architecture is included inside the click package. Feel free to experiment!
If you have any questions as usual feel free to contact me. I look forward to seeing more applications in the store from my unity8 desktop!
Friday, November 21, 2014
Virtual Hugs of appreciation!
Because I was asleep at the wheel (err, keyboard) yesterday I failed to express my appreciation for some folks. It's a day for hugging! And I missed it!
I gave everyone a shoutout on social media, but since planet looks best overrun with thank you posts, I shall blog it as well!
Thank you to:
David Planella for being the rock that has anchored the team.
Leo Arias for being super awesome and making testing what it is today on all the core apps.
Carla Sella for working tirelessly on many many different things in the years I've known her. She never gives up (even when I've tried too!), and has many successes to her name for that reason.
Nekhelesh Ramananthan for always being willing to let clock app be the guinea pig
Elfy, for rocking the manual tests project. Seriously awesome work. Everytime you use the tracker, just know elfy has been a part of making that testcase happen.
Jean-Baptiste Lallement and Martin Pitt for making some of my many wishes come true over the years with quality community efforts. Autopkgtest is but one of these.
And many more. Plus some I've forgotten. I can't give hugs to everyone, but I'm willing to try!
To everyone in the ubuntu community, thanks for making ubuntu the wonderful community it is!
I gave everyone a shoutout on social media, but since planet looks best overrun with thank you posts, I shall blog it as well!
Thank you to:
David Planella for being the rock that has anchored the team.
Leo Arias for being super awesome and making testing what it is today on all the core apps.
Carla Sella for working tirelessly on many many different things in the years I've known her. She never gives up (even when I've tried too!), and has many successes to her name for that reason.
Nekhelesh Ramananthan for always being willing to let clock app be the guinea pig
Elfy, for rocking the manual tests project. Seriously awesome work. Everytime you use the tracker, just know elfy has been a part of making that testcase happen.
Jean-Baptiste Lallement and Martin Pitt for making some of my many wishes come true over the years with quality community efforts. Autopkgtest is but one of these.
And many more. Plus some I've forgotten. I can't give hugs to everyone, but I'm willing to try!
To everyone in the ubuntu community, thanks for making ubuntu the wonderful community it is!
Labels:
ubuntu
Wednesday, November 5, 2014
Ubuntu Online Summit: Vivid Edition
Ubuntu Online Summit is once again upon us. This is a community event by and for the community. It's all encompassing and intends to cover a wide range of topics. You don't need to be a developer, project lead, member of a team, or even a member of ubuntu to join and participate. The only requirement is your passion for ubuntu and desire to discuss about it's future with others.
The dates are set as November 12-November 14th from 1400 UTC to 2000 UTC. I am once again privileged to be a track lead for the users track. In my opinion, this is the best track as it's the one the largest number of us within the community can easily feel a part of (just don't like Michael, David, Daniel or Alan know I said that). Do you use ubuntu? Awesome, this is the track for you.
What I'm asking for is sessions. Have an idea for a session? Please propose it! Everything you need to know about participating can be found here. If you've attended things like ubuntu open week or a classroom session in the past, all of those types of sessions are welcome and encouraged.
"The focus of the Users track is to highlight ways to get the most out of Ubuntu, on your laptop, your phone or your server. From detailed how-to sessions, to tips and tricks, and more, this track can provide something for everybody, regardless of skill level."
Regardless of your desire to contribute a session, I would encourage everyone to take a look at the schedule as it evolves and considering joining in sessions they find interesting. In addition, it's not yet too late to offer up ideas for sessions (though I would encourage you to find a way to host the session).
Ready to propose a session? Checkout this page and feel free to ping me or any track lead for help. Don't forget to register to attend and check out the currently scheduled sessions!
The dates are set as November 12-November 14th from 1400 UTC to 2000 UTC. I am once again privileged to be a track lead for the users track. In my opinion, this is the best track as it's the one the largest number of us within the community can easily feel a part of (just don't like Michael, David, Daniel or Alan know I said that). Do you use ubuntu? Awesome, this is the track for you.
What I'm asking for is sessions. Have an idea for a session? Please propose it! Everything you need to know about participating can be found here. If you've attended things like ubuntu open week or a classroom session in the past, all of those types of sessions are welcome and encouraged.
"The focus of the Users track is to highlight ways to get the most out of Ubuntu, on your laptop, your phone or your server. From detailed how-to sessions, to tips and tricks, and more, this track can provide something for everybody, regardless of skill level."
Regardless of your desire to contribute a session, I would encourage everyone to take a look at the schedule as it evolves and considering joining in sessions they find interesting. In addition, it's not yet too late to offer up ideas for sessions (though I would encourage you to find a way to host the session).
Ready to propose a session? Checkout this page and feel free to ping me or any track lead for help. Don't forget to register to attend and check out the currently scheduled sessions!
Labels:
opportunity,
ubuntu
Tuesday, November 4, 2014
Autopilot Feature Primer
With this in mind, I thought it would be useful to bring attention to some new and under-used features of autopilot, along with providing a brief explanation of some companion tools you might find useful. Thus I present to you, an autopilot primer. Let's talk through some new features shall we?
Python3 Support
Autopilot started as a python2 tool but has since migrated to python3 and you should too! For now the entire source tree remains python2 compatible, but you really should migrate your tests to python3. You'll notice the autopilot3 binary in newer releases which should be used to run autopilot with python3.
Scenario Support
Scenarios are a wonderful way to keep your tests simple and easy to read while allowing you to test with many different inputs. In short, you might need to test several edge cases as part of your acceptance testing. This is most easily accomplished by keeping the test itself generic and utilizing a scenario to vary your inputs. You can check out more information on scenarios specific to autopilot in the autopilot documentation.
Screenshots / Video
Autopilot allows you to get a video recording of a test failure. To make sure autopilot records failures, install recordmydesktop and pass the -r argument to your autopilot3 run command. However, at the moment this requires X so for now it doesn't work with the MIR backend (which things like the ubuntu phone utilize). Fortunately a screenshot at the point of failure when combined with the log is generally sufficient to solving your issue. Getting those screenshots brings us to subunit support.
Subunit Support
By default autopilot generates the test output and logs straight to your console in a text format. However autopilot also supports outputting to xml and subunit. Subunit support is what I would like to highlight for a few reasons. When you set the output format as subunit, you get a few niceties. One of which is an easier to grok format for tools, and the other is screenshots of the application when failures occur. To get a subunit stream, pass the -f subunit argument to your autopilot3 run command. You will want to also pass -O with a filename to save the output to a file as the subunit stream contains binary data.
Test Result Viewer
So, with this subunit test results file, how can you enjoy all of it's goodness? Enter trv, a simple python ui that will let organize the test run in an easy to view manner, including screenshots. The tool is the creation of Thomi Richards who describes it as a quick hack (:p), and has a youtube video demonstrating it's use. It's perfect for viewing the subunit stream and visualizing your test results. For now, it's not packaged but can be easily obtained via launchpad. Grab it with bzr branch lp:trv.
autopilot3 vis
The vis tool allows you to visually interact with the introspection tree after launching an application using autopilot launch. What this means is you can visualize the application in the same way autopilot does at runtime, with live tree updates. It lets you see what autopilot sees, allowing you to interactively build your testcase.
I'll refer you the official tutorial for more information, as well as a youtube video by yours truly. It's from a livestream, but covers what you need to know. autopilot3 vis also contains a search box, and a highlight tool which didn't exist in the orignal version, so it's even nicer now than before. Give it a whirl!
autopilot3-sandbox-run
I talked about this utility when I covered the test runners for autopilot. Still I would be remiss if I didn't mention it again. Everything I said in the test runners for autopilot post still applies, so go have a quick read about how to use the tool if you need more information. This tool enables you to easily run autopilot tests on the desktop in a nested xserver. What that means to you as a test author is that you can run tests without giving up your desktop session. No more waiting for autopilot to hand back control of your mouse after a test. If you are writing tests, you should be using this tool along with autopilot vis mentioned above during your test writing process.
Per test timeout
Although we all only write "good tests", sometimes you may find your test misbehaves. When this happens the test may even not exit cleanly or get stuck in a loop. The result is autopilot and the system under test to wait forever for the test to finish. To prevent a rouge test from killing a test suite run, autopilot is introducing support for per-test timeouts. This has landed in vivid; you'll need version 1.5.0+15.04.20141031-0ubuntu1 or later. To use the feature, add the --test-timeout argument to autopilot run and give is a timeout in seconds.
In conclusion
Autopilot has gotten many new features along the way, and these are but a few of the most recent and important ones. I hope this helps you take another look at what autopilot might be able to help you test. Happy Testing!
Monday, October 27, 2014
Sprinting in DC: Friday
This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.
Friday brings an end to an exciting week, and the faces of myself and those around me reflect the discussions, excitement, fun and lack of sleep this week has entailed.
The first session of the day involved hanging out with the QA team while they heard feedback from various teams on issues with quality and process within there project. Always fun to hear about what causes different teams the most issues when it comes to testing.
Next I spent some time interviewing a couple folks for publishing later. In my case I interviewed Thomi from the QA team and Zoltan from the SDK team about the work going on within there teams and how the last cycle went. The team as a whole has been conducting interviews all week. Look for these interviews to appear on youtube in the coming weeks.
Thursday night while having a look through a book store, I came across an ad for ubuntu in Linux Voice magazine. It made me smile. The dream of running ubuntu on all my devices is becoming closer every day.
I'd like to thank all the community core app developers who joined us this week. Thanks for hanging out with us, providing feedback, and most of all for the creating the wonderful apps we have for the ubuntu phone. Your work has helped shaped the device and turn it into what it is today.
Looking back over the schedule there were sessions I wish I had been able to attend, and it was wonderful catching up with everyone. Sadly my flight home prevented me from attending the closing session and presumably getting a summary of some of these sessions. I can say I was delighted to talk and interact with the unity8 team on the next steps for unity8 on the desktop. I trust next cycle we as a community can do more around testing there work.
As I head to the airport for home, it's time to celebrate the release of utopic unicorn!
Friday brings an end to an exciting week, and the faces of myself and those around me reflect the discussions, excitement, fun and lack of sleep this week has entailed.
 |
| Bubbles! |
Next I spent some time interviewing a couple folks for publishing later. In my case I interviewed Thomi from the QA team and Zoltan from the SDK team about the work going on within there teams and how the last cycle went. The team as a whole has been conducting interviews all week. Look for these interviews to appear on youtube in the coming weeks.
Thursday night while having a look through a book store, I came across an ad for ubuntu in Linux Voice magazine. It made me smile. The dream of running ubuntu on all my devices is becoming closer every day.
I'd like to thank all the community core app developers who joined us this week. Thanks for hanging out with us, providing feedback, and most of all for the creating the wonderful apps we have for the ubuntu phone. Your work has helped shaped the device and turn it into what it is today.
Looking back over the schedule there were sessions I wish I had been able to attend, and it was wonderful catching up with everyone. Sadly my flight home prevented me from attending the closing session and presumably getting a summary of some of these sessions. I can say I was delighted to talk and interact with the unity8 team on the next steps for unity8 on the desktop. I trust next cycle we as a community can do more around testing there work.
As I head to the airport for home, it's time to celebrate the release of utopic unicorn!
Friday, October 24, 2014
Sprinting in DC: Thursday
This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.
Today started with some UOS planning which is happening in a couple short weeks. If you haven't yet put it on your calendar, please do so! And plan to not only attend, but consider submitting a session as well. The users track might be just the place for your session. Session topics can be about anything ubuntu related you might want to share or discuss with others.
As the week has progressed I've enjoyed getting to know the core apps developers better. Today we met with all of them to hear feedback on how the projects have been going. Lots of good discussion was had discussing how things like meetings and reviews work, individual project needs and actions that could be taken to improve all of the projects. It's wonderful to have everyone in the same place and able to talk.
After lunch the QA team discussed manual testing and proposed utilizing moztrap for some of the manual testing they are undertaking as part of the CI process for ubuntu touch images. While it is too early to say what implications this will have on manual testing from a community perspective, I'm happy to see the conversation has begun around the current issues facing manual tests. I'm also happy someone else is willing to be a guinea pig for changes like this! For image testing, the qatracker has served us well and will continue to do so, but I hope in the future we can improve the experience. In fact, we have done work in this area recently, and would love to hear from anyone who wants to help improve the qatracker experience. So, whether or not a migration to moztrap occurs at some point, the future looks bright.
The core app developers also got a chance to both get and receive feedback from the SDK and design teams. The deep dives into applications like calendar were very much appreciated and I expect those suggestions will filter into the applications in the near future. As usual the core apps developers came prepared with suggestions and grievances for the SDK team, as well as praises for things done well.
Finally to end the day, we discussed developer mode on the device. Rather than talk about the history of how it was implemented, let me share with you the future. Rather than locking adb access via a password, we'll utilize certificates. The password based solution already will ensure your locked device isn't vulnerable to nefarious humans who might want to connect and steal your data or reflash your phone. However, things like passwordless sudo will be possible with using certificates. In addition if security is the bane of your existence, you will be able to enable developer mode without setting a password at all.
Whew, today was very full!
Today started with some UOS planning which is happening in a couple short weeks. If you haven't yet put it on your calendar, please do so! And plan to not only attend, but consider submitting a session as well. The users track might be just the place for your session. Session topics can be about anything ubuntu related you might want to share or discuss with others.
As the week has progressed I've enjoyed getting to know the core apps developers better. Today we met with all of them to hear feedback on how the projects have been going. Lots of good discussion was had discussing how things like meetings and reviews work, individual project needs and actions that could be taken to improve all of the projects. It's wonderful to have everyone in the same place and able to talk.
After lunch the QA team discussed manual testing and proposed utilizing moztrap for some of the manual testing they are undertaking as part of the CI process for ubuntu touch images. While it is too early to say what implications this will have on manual testing from a community perspective, I'm happy to see the conversation has begun around the current issues facing manual tests. I'm also happy someone else is willing to be a guinea pig for changes like this! For image testing, the qatracker has served us well and will continue to do so, but I hope in the future we can improve the experience. In fact, we have done work in this area recently, and would love to hear from anyone who wants to help improve the qatracker experience. So, whether or not a migration to moztrap occurs at some point, the future looks bright.
The core app developers also got a chance to both get and receive feedback from the SDK and design teams. The deep dives into applications like calendar were very much appreciated and I expect those suggestions will filter into the applications in the near future. As usual the core apps developers came prepared with suggestions and grievances for the SDK team, as well as praises for things done well.
Finally to end the day, we discussed developer mode on the device. Rather than talk about the history of how it was implemented, let me share with you the future. Rather than locking adb access via a password, we'll utilize certificates. The password based solution already will ensure your locked device isn't vulnerable to nefarious humans who might want to connect and steal your data or reflash your phone. However, things like passwordless sudo will be possible with using certificates. In addition if security is the bane of your existence, you will be able to enable developer mode without setting a password at all.
Whew, today was very full!
Thursday, October 23, 2014
Sprinting in DC: Wednesday
This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.
To kick off the day, I led a session on something that has been wreaking havoc for application test writers within the core apps -- environment setup. In theory, setting up the environment to run your test should be easy. In practice, I've found it increasingly difficult. The music, calendar, clock, reminders, file manager and other teams have all been quite affected by this and the canonical QA team and myself have all pitched in to help, but struggled as well. In short, a test should be easy to launch, be well behaved and not delete any user data, and be easy to setup and feed test data into for the test process. I'm happy to report that the idea of a permanent solution has been reached. Now we must implement it of course, but the result should be drastically easier and more reliable test setup for you the test author.
I also had the chance to list some grievances for application developers with the QA team. We spoke about wanting to expand the documentation on testing and specifically targeted the need to create better templates in the ubuntu sdk for new projects. When you start a new project you should have well functioning tests, and we should teach you about how to run them too!
Just before lunch the community core app developers were able to discuss post-RTM plans and features. A review of the apps was undertaken and some desire for new designs or features were discussed. Terminal is being rebuilt to be more aligned with upstream. Music is currently undergoing a re-design which is coming along great. Calculator is anxious to get some design love. Reminders potential for offline notetaking as well as potential name changes were all discussed. Overall, an amazing accomplishment by all the developers!
After lunch, I spent time confirming the fix for a longstanding bug within autopilot. The merge proposal for fixing this bug has been simmering all summer and it's time to get it fixed. The current test suites for calendar and clock have been impacted by this and have already had regressions occur that could have been caught had tests been able to be written for this area. Having myself, the autopilot team, and the calendar developers in one place made fixing this possible.
To end the day, I spent some time attending sessions for changes to CI and learning more about the coming changes to CI within ubuntu. In summary the news is wonderful. CI will test using autopkgtest, and all of ubuntu will come under this umbrella -- phone, desktop, everything. If it's a package and it has tests, we will do all of the autopkgtest goodness currently being done for the distro.
The evening closed with a bit of fun provided by a game making hackathon using bacon2d and the hilariously horrible "Turkish Star Wars". We could always use more games in the ubuntu app store, and I hear there might even still be a pioneers t-shirt or two left if you get it in early!
To kick off the day, I led a session on something that has been wreaking havoc for application test writers within the core apps -- environment setup. In theory, setting up the environment to run your test should be easy. In practice, I've found it increasingly difficult. The music, calendar, clock, reminders, file manager and other teams have all been quite affected by this and the canonical QA team and myself have all pitched in to help, but struggled as well. In short, a test should be easy to launch, be well behaved and not delete any user data, and be easy to setup and feed test data into for the test process. I'm happy to report that the idea of a permanent solution has been reached. Now we must implement it of course, but the result should be drastically easier and more reliable test setup for you the test author.
I also had the chance to list some grievances for application developers with the QA team. We spoke about wanting to expand the documentation on testing and specifically targeted the need to create better templates in the ubuntu sdk for new projects. When you start a new project you should have well functioning tests, and we should teach you about how to run them too!
Just before lunch the community core app developers were able to discuss post-RTM plans and features. A review of the apps was undertaken and some desire for new designs or features were discussed. Terminal is being rebuilt to be more aligned with upstream. Music is currently undergoing a re-design which is coming along great. Calculator is anxious to get some design love. Reminders potential for offline notetaking as well as potential name changes were all discussed. Overall, an amazing accomplishment by all the developers!
After lunch, I spent time confirming the fix for a longstanding bug within autopilot. The merge proposal for fixing this bug has been simmering all summer and it's time to get it fixed. The current test suites for calendar and clock have been impacted by this and have already had regressions occur that could have been caught had tests been able to be written for this area. Having myself, the autopilot team, and the calendar developers in one place made fixing this possible.
To end the day, I spent some time attending sessions for changes to CI and learning more about the coming changes to CI within ubuntu. In summary the news is wonderful. CI will test using autopkgtest, and all of ubuntu will come under this umbrella -- phone, desktop, everything. If it's a package and it has tests, we will do all of the autopkgtest goodness currently being done for the distro.
The evening closed with a bit of fun provided by a game making hackathon using bacon2d and the hilariously horrible "Turkish Star Wars". We could always use more games in the ubuntu app store, and I hear there might even still be a pioneers t-shirt or two left if you get it in early!
Wednesday, October 22, 2014
Sprinting in DC: Tuesday
This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.
On Tuesday I was finally able to sit down with the team and plan our week. In addition I was able to plan some of the work I had in mind with the community folks working on the core apps. Being obsessed with testing, my primary goals this week are centered around quality. Namely I want to make it easier for developers to write tests. Asking them to write tests is much easier when it's easy to do so. Fortunately, I think (hope?) all of the community core apps developers recognize the benefits to tests and thus are motivated to drive maturity into the testing story.
I'm also keen to work on the manual testing story. The community is imperative in helping test images for not only ubuntu, but also all of it's flavors. Seriously, you should say thank you to those folks helping make sure your install of ubuntu works well. They are busy this week helping make sure utopic is as good as it can be. Rock on image testers! But the tools and process used weigh on my mind, and I'm keen to chat later in the week with the canonical QA team and get there feedback.
During the day I attended sessions regarding changes and tweaks to the CI process. For core apps developers, errors in jenkins should be easier to replicate after these changes. CI will be moving to utilizing adt-run (autopkgtest) for there test execution (and you should too!). They will also provide the exact commands used to run the test. That means you can easily duplicate the results on the dashboard locally and fix the issues found. No more works on my box excuses!
I also met the team responsible for the application store and gave them feedback on the application submission process. Submitting apps is already so simple, but even more cool things are happening on this front.
The end of the evening found us shuffling into cab's for a team dinner. We had a long table of folks eating Italian food and getting to know each other better.
After dinner, I pressured a few folks into having some dessert and ordered a sorbet for myself. After receiving no less than 4 fruit sorbets due to a misunderstanding, I began carving the fruits and sending plates of sorbet down the table. My testcase failed however when the plates all came back :-(
On Tuesday I was finally able to sit down with the team and plan our week. In addition I was able to plan some of the work I had in mind with the community folks working on the core apps. Being obsessed with testing, my primary goals this week are centered around quality. Namely I want to make it easier for developers to write tests. Asking them to write tests is much easier when it's easy to do so. Fortunately, I think (hope?) all of the community core apps developers recognize the benefits to tests and thus are motivated to drive maturity into the testing story.
I'm also keen to work on the manual testing story. The community is imperative in helping test images for not only ubuntu, but also all of it's flavors. Seriously, you should say thank you to those folks helping make sure your install of ubuntu works well. They are busy this week helping make sure utopic is as good as it can be. Rock on image testers! But the tools and process used weigh on my mind, and I'm keen to chat later in the week with the canonical QA team and get there feedback.
During the day I attended sessions regarding changes and tweaks to the CI process. For core apps developers, errors in jenkins should be easier to replicate after these changes. CI will be moving to utilizing adt-run (autopkgtest) for there test execution (and you should too!). They will also provide the exact commands used to run the test. That means you can easily duplicate the results on the dashboard locally and fix the issues found. No more works on my box excuses!
I also met the team responsible for the application store and gave them feedback on the application submission process. Submitting apps is already so simple, but even more cool things are happening on this front.
The end of the evening found us shuffling into cab's for a team dinner. We had a long table of folks eating Italian food and getting to know each other better.
Sprinting in DC: Monday
This week, my team and I are sprinting in Washington DC with many of the core app developers and other folks inside of Ubuntu Engineering. Sprints are always busy, but the work tends to be a mix of social and technical. I get to assign names (IRC nicknames mostly) to faces as well as get to know my co-workers and other community members better.
I thought it might be useful to give writeups each day of what's going on, at least from my perspective during the sprint. I won't yammer on too much about quality and instead bring you pictures of what you really want. And some of this too. Whoops, here's one.
Monday was the first day of the sprint, and also the day of my arrival! Personally I'm busy at home during this week, so it's tough to get away. That said, I can't imagine being anywhere else for the week. The sprints are a wonderful source of respite for everyone.
Monday itself consisted of making sure everything is ready for the week, planning events, and icebreakers. In typical fashion, an opening plenary set the bar for the week with notes about the progress being made on the phone as well as the future of the desktop. Lots of meetings and a few blurry jet lagged hours later, everyone was ready to sit for a bit and have some non-technical conversation!
Fortunately for us there was an event planned to meet both our social and hunger needs. After being split randomly into teams of bugs (love the play on quality), we played a bit of trivia. After each round teams were scored not only on the correct response, but also how quickly they responded. The questions varied from the obscure to fun bits about ubuntu. The final round centered around Canonical itself which was fun trip down memory lane to remember.
As I crawled into bed I still had the wonderfully cheesy announcer playing trivia questions in my head.
I thought it might be useful to give writeups each day of what's going on, at least from my perspective during the sprint. I won't yammer on too much about quality and instead bring you pictures of what you really want. And some of this too. Whoops, here's one.
 |
| Pictures of people taking pictures . . . |
Monday itself consisted of making sure everything is ready for the week, planning events, and icebreakers. In typical fashion, an opening plenary set the bar for the week with notes about the progress being made on the phone as well as the future of the desktop. Lots of meetings and a few blurry jet lagged hours later, everyone was ready to sit for a bit and have some non-technical conversation!
Fortunately for us there was an event planned to meet both our social and hunger needs. After being split randomly into teams of bugs (love the play on quality), we played a bit of trivia. After each round teams were scored not only on the correct response, but also how quickly they responded. The questions varied from the obscure to fun bits about ubuntu. The final round centered around Canonical itself which was fun trip down memory lane to remember.
As I crawled into bed I still had the wonderfully cheesy announcer playing trivia questions in my head.
Thursday, October 16, 2014
Final testing for Utopic
The final images of what will become utopic are here! Yes, in just one short week utopic unicorn will be released into the world. Celebrate this exciting release and be among the first to run utopic by helping us test!
We need your help and test results, both positive and negative. Please head over to the milestone on the isotracker, select your favorite flavor, and perform the needed tests against the images.
If you've never submitted test results for the iso tracker, check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help.
Thank you for helping to make ubuntu better! Happy Testing!
We need your help and test results, both positive and negative. Please head over to the milestone on the isotracker, select your favorite flavor, and perform the needed tests against the images.
If you've never submitted test results for the iso tracker, check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help.
Thank you for helping to make ubuntu better! Happy Testing!
Labels:
call for testing,
opportunity,
quality,
testing
Tuesday, September 23, 2014
Final Beta testing for Utopic
Can you believe final beta is here for utopic already? Where has the summer gone? The milestone and images are already prepared for the final beta testing. This is the first round of image testing for ubuntu this cycle. A final image will also be tested next month, but now is the time to try out the image on your system. Be sure to report any bugs you may find. This will help ensure there is time to fix them before the release images.
To help make sure the final utopic image is in good shape, we need your help and test results! Please, head over to the milestone on the isotracker, select your favorite flavor and perform the needed tests against the images.
If you've never submitted test results for the iso tracker, check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help. Happy Testing!
To help make sure the final utopic image is in good shape, we need your help and test results! Please, head over to the milestone on the isotracker, select your favorite flavor and perform the needed tests against the images.
If you've never submitted test results for the iso tracker, check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help. Happy Testing!
Labels:
call for testing,
quality,
ubuntu
Thursday, September 4, 2014
Autopilot Test Runners
In the meantime I would be remiss if I didn't also talk about the different test runners commonly used with autopilot tests. In addition to the autopilot binary which can be executed to run the tests, different tools have cropped up to make running tests easier.
autopilot-sandbox-run
This tool ships with autopilot itself and was developed as a way to run autopilot test suites on your desktop in a sane manner. Run the autopilot3-sandbox-run command with --help to see all the options available. By default, the tests will run in an Xvfb server, all completely behind the scenes with the results being reported to you upon completion. This is a great way to run tests with no interference on your desktop. If you are a visual person like me, you may instead wish to pass -X to enable the test runs to occur in a Xephyr window allowing you to see what's happening, but still retaining control of your mouse and keyboard.
I need this tool!
sudo apt-get install python3-autopilot
I want to run tests on my desktop without losing control of my mouse!
autopilot3-sandbox-run my_testsuite_name
I want to run tests on my desktop without losing control of my mouse, but I still want to see what's happening!
autopilot3-sandbox-run -X my_testsuite_name
Autopkgtest
Autopkgtest was developed as a means to automatically test Debian packages, "as-installed". Recently support was added to also test click packages and to run on phablet devices. Autopkgtest will take care of dependencies, setting up autopilot, and unlocking the device. You can literally plug in a device and wait for the results. You should really checkout the README pages, including those on running tests. That said, here's a quick primer on running tests using autopkgtest.
I need this tool!
sudo apt-get install autopkgtest
If you are on trusty, grab and install the utopic deb from here.
I want to run tests for a click package installed on my device!
Awesome. This one is simple. Connect the device and then run:
adt-run --click my.click.name --- ssh -s adb
For example,
adt-run --click com.ubuntu.music --- ssh -s adb
will run the tests for the installed version of the music app on your device. You don't need to do anything else. For the curious, this works by reading the manifest file all click packages have. Read more here.
I want to run the tests I wrote/modified against an installed click package!
For this you need to also pass your local folder containing the tests. You will also want to make sure you installed the new version of the click package if needed.
adt-run my-folder/ --click my.click.name --- ssh -s adb
Autopkgtest can also run in a lxc container, QEMU, a chroot, and other fun targets. In the examples above, I passed --- ssh -s adb as the target, instructing autopkgtest to use ssh and adb and thus run the tests on a connected phablet device. If you want to run autopilot tests on a phablet device, I recommend using autopkgtest as it handles everything for you.
phablet-test-run
This tool is part of the greater phablet-tools package. It was originally developed as an easy way to execute tests on your phablet device. Note however that copying the tests and any dependencies to the phablet device is left to you. The phablet-tools package provides some other useful utilities to help you with this (checkout phablet-click-test-setup for example).
I need this tool!
sudo apt-get install phablet-tools
I want to run the tests I wrote/modified against an installed click package!
First copy the tests to the device. You can use the ubuntu sdk or click-buddy for this, or even do it manually via adb. Then run phablet-test-run. It takes the same arguments as autopilot itself.
phablet-test-run -v my_testsuite
Note the tools looks for the testsuite and any dependencies of the testsuite inside the /home/phablet/autopilot folder. It's up to you to make sure everything that is needed to run your tests are located there or else it will fail.
other ways
There are of course other possible test runners that wrap around autopilot to make executing tests easier. Perhaps you've written a script yourself. Just remember at the end of the day the autopilot binary will be running the tests. It simply needs to be able to find the testsuite and all of it's dependencies in order to run. For this reason, don't be afraid to execute autopilot3 and run the tests yourself. Happy test runs!
Monday, August 25, 2014
Ubuntu Phone Translations Needed
As we continue to iterate on new ubuntu touch images, it's important for everyone to be able to enjoy the ubuntu phone experience in their native language. This is where you can help!
We need your input and help to make sure the phone images are well localized for your native language. If you've never contributed a translation before, this is a perfect opportunity for you to learn. There's a wiki guide to help you, along with translation teams who speak your language and can help.
Don't worry, you don't need a ubuntu phone to do this work. The wiki guide details how to translate using a phone, emulator, or even just your desktop PC running ubuntu. If nothing else, you can help review other folks translations by simply using launchpad in your web browser.
If this sounds interesting to you and the links don't make sense or you would like some more personal help, feel free to contact me. English is preferred, but in the spirit of translation feel free to contact me in French, Spanish or perhaps even German :-).
Happy Translating everyone!
P.S. If you are curious about the status of your language translation, or looking for known missing strings, have a look at the stats page kept by David Planella.
We need your input and help to make sure the phone images are well localized for your native language. If you've never contributed a translation before, this is a perfect opportunity for you to learn. There's a wiki guide to help you, along with translation teams who speak your language and can help.
Don't worry, you don't need a ubuntu phone to do this work. The wiki guide details how to translate using a phone, emulator, or even just your desktop PC running ubuntu. If nothing else, you can help review other folks translations by simply using launchpad in your web browser.
If this sounds interesting to you and the links don't make sense or you would like some more personal help, feel free to contact me. English is preferred, but in the spirit of translation feel free to contact me in French, Spanish or perhaps even German :-).
Happy Translating everyone!
P.S. If you are curious about the status of your language translation, or looking for known missing strings, have a look at the stats page kept by David Planella.
Labels:
call for testing,
opportunity
Wednesday, July 16, 2014
A new test runner approaches
The problem
How acceptance tests are packaged and run has morphed over time. When autopilot was originally conceived the largest user was the unity project and debian packaging was the norm. Now that autopilot has moved well beyond that simple view to support many types of applications running across different form factors, it was time to address the issue of how to run and package these high-level tests.
While helping develop testsuites for the core apps targeting ubuntu touch, it became increasingly difficult for developers to run their application's testsuites. This gave rise to further integration points inside qtcreator, enhancements to click and its manifest files, and tools like the phablet-tools suite and click-buddy. All of these tools operate well within the confines they are intended, but none truly meets the needs for test provisioning and execution.
A solution?
With these thoughts in mind I opened the floor for discussion a couple months ago detailing the need for a proper tool that could meet all of my needs, as well as those of the application developer, test author and CI folks. In a nutshell, a workflow to setup a device as well as properly manage dependencies and resolve them was needed.
Autopkg tests all the things
I'm happy to report that as of a couple weeks ago such a tool now exists in autopkgtest. If the name sounds familar, that's because it is. Autopkgtest already runs all of our automated testing at the archive level. New package uploads are tested utilizing its toolset.
So what does this mean? Utilizing the format laid out by autopkgtest, you can now run your autopilot testsuite on a phablet device in a sane manner. If you have test dependencies, they can be defined and added to the click manifest as specified. If you don't have any test dependencies, then you can run your testsuite today without any modifications to the click manifest.
Yes, but what does this really mean?
This means you can now run a testsuite with adt-run in a similar manner to how debian packages are tested. The runner will setup the device, copy the tests, resolve any dependencies, run them, and report the results back to you.
Some disclaimers
Support for running tests this way is still new. If you do find a bug, please file it!
To use the tool first install autopkgtest. If you are running trusty, the version in the archive is old. For now download the utopic deb file and install it manually. A proper backport still needs to be done.
Also as of this writing, I must caution you that you may run into this bug. If the application fails to download dependencies (you see 404 errors during setup), update your device to the latest image and try again. Note, the latest devel image might be too old if a new image hasn't been promoted in a long time.
I want to see it!
Go ahead, give it a whirl with the calendar application (or your favorite core app). Plug in a device, then run the following on your pc.
bzr branch lp:ubuntu-calendar-app
adt-run ubuntu-calendar-app --click=com.ubuntu.calendar --- ssh -s /usr/share/autopkgtest/ssh-setup/adb
Autopkgtest will give you some output along the way about what is happening. The tests will be copied, and since --click= was specified, the runner will use the click from the device, install the click in our temporary environment, and read the click manifest file for dependencies and install those too. Finally, the tests will be executed with the results returned to you.
Feedback please!
Please try running your autopilot testsuites this way and give feedback! Feel free to contact myself, the upstream authors (thanks Martin Pitt for adding support for this!), or simply file a bug. If you run into trouble, utilize the -d and the --shell switches to get more insight into what is happening while running.
How acceptance tests are packaged and run has morphed over time. When autopilot was originally conceived the largest user was the unity project and debian packaging was the norm. Now that autopilot has moved well beyond that simple view to support many types of applications running across different form factors, it was time to address the issue of how to run and package these high-level tests.
While helping develop testsuites for the core apps targeting ubuntu touch, it became increasingly difficult for developers to run their application's testsuites. This gave rise to further integration points inside qtcreator, enhancements to click and its manifest files, and tools like the phablet-tools suite and click-buddy. All of these tools operate well within the confines they are intended, but none truly meets the needs for test provisioning and execution.
A solution?
With these thoughts in mind I opened the floor for discussion a couple months ago detailing the need for a proper tool that could meet all of my needs, as well as those of the application developer, test author and CI folks. In a nutshell, a workflow to setup a device as well as properly manage dependencies and resolve them was needed.
Autopkg tests all the things
I'm happy to report that as of a couple weeks ago such a tool now exists in autopkgtest. If the name sounds familar, that's because it is. Autopkgtest already runs all of our automated testing at the archive level. New package uploads are tested utilizing its toolset.
So what does this mean? Utilizing the format laid out by autopkgtest, you can now run your autopilot testsuite on a phablet device in a sane manner. If you have test dependencies, they can be defined and added to the click manifest as specified. If you don't have any test dependencies, then you can run your testsuite today without any modifications to the click manifest.
Yes, but what does this really mean?
This means you can now run a testsuite with adt-run in a similar manner to how debian packages are tested. The runner will setup the device, copy the tests, resolve any dependencies, run them, and report the results back to you.
Some disclaimers
Support for running tests this way is still new. If you do find a bug, please file it!
To use the tool first install autopkgtest. If you are running trusty, the version in the archive is old. For now download the utopic deb file and install it manually. A proper backport still needs to be done.
Also as of this writing, I must caution you that you may run into this bug. If the application fails to download dependencies (you see 404 errors during setup), update your device to the latest image and try again. Note, the latest devel image might be too old if a new image hasn't been promoted in a long time.
I want to see it!
Go ahead, give it a whirl with the calendar application (or your favorite core app). Plug in a device, then run the following on your pc.
bzr branch lp:ubuntu-calendar-app
adt-run ubuntu-calendar-app --click=com.ubuntu.calendar --- ssh -s /usr/share/autopkgtest/ssh-setup/adb
Autopkgtest will give you some output along the way about what is happening. The tests will be copied, and since --click= was specified, the runner will use the click from the device, install the click in our temporary environment, and read the click manifest file for dependencies and install those too. Finally, the tests will be executed with the results returned to you.
Feedback please!
Please try running your autopilot testsuites this way and give feedback! Feel free to contact myself, the upstream authors (thanks Martin Pitt for adding support for this!), or simply file a bug. If you run into trouble, utilize the -d and the --shell switches to get more insight into what is happening while running.
Monday, July 14, 2014
Utopic Test Writing Hackfest
We're having our first hackfest of the utopic cycle this week on Tuesday, July
15th. You can catch us live in a hangout on ubuntuonair.com starting at 1900 UTC. Everything you need to know can be found on the wiki page for the event.
During the hangout, we'll be demonstrating writing a new manual testcase, as well as reviewing writing automated testcases. We'll be answering any questions you have as well about contributing a testcase.
We need your help to write some new testcases! We're targeting both manual and automated testcase, so everyone is welcome to pitch in.
We are looking at writing and finishing some testcases for ubuntu studio and some other flavors. All you need is some basic tester knowledge and the ability to write in English.
If you know python, we are also going to be hacking on the toolkit helper for autopilot for the ubuntu sdk. That's a mouthful! Specifically it's the helpers that we use for writing autopilot tests against ubuntu-sdk applications. All app developers make use of these helpers, and we need more of them to ensure we have good coverage for all components developers use.
Don't worry about getting stuck, we'll be around to help, and there's guides to well, guide you!
Hope to see everyone there!
During the hangout, we'll be demonstrating writing a new manual testcase, as well as reviewing writing automated testcases. We'll be answering any questions you have as well about contributing a testcase.
We need your help to write some new testcases! We're targeting both manual and automated testcase, so everyone is welcome to pitch in.
We are looking at writing and finishing some testcases for ubuntu studio and some other flavors. All you need is some basic tester knowledge and the ability to write in English.
If you know python, we are also going to be hacking on the toolkit helper for autopilot for the ubuntu sdk. That's a mouthful! Specifically it's the helpers that we use for writing autopilot tests against ubuntu-sdk applications. All app developers make use of these helpers, and we need more of them to ensure we have good coverage for all components developers use.
Don't worry about getting stuck, we'll be around to help, and there's guides to well, guide you!
Hope to see everyone there!
Labels:
autopilot,
opportunity,
quality,
testing
Tuesday, July 8, 2014
Utopic Bug Hug and Testing Day
The first testing day of the utopic cycle is coming this week on Thursday, July
10th. You can catch us live in a hangout on ubuntuonair.com starting at 1900 UTC. We'll be demonstrating running and
testing the development release of ubuntu, reporting test results, reporting bugs, and doing triage work. We'll also be availible to answer your questions and help you get started testing as well.
Please join us in testing utopic and helping the next release of ubuntu become the best it can be. Hope to see everyone there!
P.S. We have a team calendar that can help you keep track of the release schedule along with this and other events. Check it out!
Please join us in testing utopic and helping the next release of ubuntu become the best it can be. Hope to see everyone there!
P.S. We have a team calendar that can help you keep track of the release schedule along with this and other events. Check it out!
Labels:
opportunity,
quality,
testing
Monday, June 2, 2014
Calling for your UOS users session!
Ubuntu Online Summit is approaching, happening on June 10th-12th.This time it's a bit different from how vUDS has been in the past. Rather than the narrower developer focus, this intends to be a full blown community summit. If you've attending things like ubuntu open week or a classroom session in the past, all of those types of sessions are welcome and encouraged too.
To help foster these types of sessions, there is a special Users track.
"The focus of the Users track is to highlight ways to get the most out of Ubuntu, on your laptop, your phone or your server. From detailed how-to sessions, to tips and tricks, and more, this track can provide something for everybody, regardless of skill level."
Track Leads:
Elizabeth Krumbach Joseph
Nicholas Skaggs
Valorie Zimmerman
I'm excitied to be a track lead for this track along with Liz and Val. We are all inviting you to consider scheduling a session to share your knowledge of ubuntu. Share an idea, discuss your passion, give a how-to, etc. The sessions in this track are meant for other users of ubuntu like yourself, so feel free to share.
Regardless of your desire to contribute a session, I would encourage everyone to take a look at the schedule as it evolves and considering joining in sessions they find interesting.
.
Remember, this track is your track and filled with your sessions. Let's help make the online summit a success.
So ready to propose a session? Checkout this page and feel free to ping Val, Liz or myself for help. Don't forget to register to attend and check out the currently scheduled sessions!
To help foster these types of sessions, there is a special Users track.
"The focus of the Users track is to highlight ways to get the most out of Ubuntu, on your laptop, your phone or your server. From detailed how-to sessions, to tips and tricks, and more, this track can provide something for everybody, regardless of skill level."
Track Leads:
Elizabeth Krumbach Joseph
Nicholas Skaggs
Valorie Zimmerman
I'm excitied to be a track lead for this track along with Liz and Val. We are all inviting you to consider scheduling a session to share your knowledge of ubuntu. Share an idea, discuss your passion, give a how-to, etc. The sessions in this track are meant for other users of ubuntu like yourself, so feel free to share.
Regardless of your desire to contribute a session, I would encourage everyone to take a look at the schedule as it evolves and considering joining in sessions they find interesting.
.
Remember, this track is your track and filled with your sessions. Let's help make the online summit a success.
So ready to propose a session? Checkout this page and feel free to ping Val, Liz or myself for help. Don't forget to register to attend and check out the currently scheduled sessions!
Labels:
opportunity
Thursday, May 1, 2014
Building cross-architecture click packages
Building click packages should be easy. And to a reasonable extent, qtcreator and click-buddy do make it easy. Things however can get a bit more complicated when you need to build a package that needs to run on an armhf device (you know like your phone!). Since your pc is almost certainly based on x86, you need to use, create or fake an armhf environment for building the package.
So then what options exist for getting a proper build of a project that will install properly on your device?
A phone can be more than a phone
It can also be a development environment!? Although it's not my recommendation, you can always use the source device to compile the package with. The downsides of this is namely speed and storage space. Nevertheless, it will build a click.
The click tools contain a handy way to build a chroot expressly suited for use with click-buddy to build things. Basically, we can create a nice fake environment and pretend it's armhf, even though we're not running that architecture.
sudo click chroot -a armhf -f ubuntu-sdk-14.04 create
click-buddy --dir . --arch armhf
Most likely your package will require extra dependencies, which for now will need to be specified and passed in with the --extra-deps argument. These arguments are packages names, just like you would apt-get. Like so;
click-buddy --dir . --arch armhf --extra-deps "libboost-dev:armhf libssl-dev:armhf"
Notice we specified the arch as well, armhf. If we also add a --maint-mode, our extra installed packages will persist. This is handy if you will only ever be building a single project and don't want to constantly update the base chroot with your build dependencies.
Qtcreator build it for me!
Cmake makes all things possible. Qt Creator can not only build the click for you, it can also hold your hand through creating a chroot1. To create a chroot in qtcreator, do the following:
So, click can setup a chroot for you, and qt creator can build and manage one too. And these are great options for building one project. However if you find yourself building a plethora of packages or you simply want more control, I recommend setting up and using your own chroot to build. For my own use, I've picked pbuilder, but you can setup the chroot using other tools (like schroot which Qt Creator uses).
sudo apt-get install qemu-user-static ubuntu-dev-tools
pbuilder-dist trusty armhf create
pbuilder-dist trusty armhf login --save-after-login
Then, from inside the chroot shell, install a couple things you will always want available; namely the build tools and bzr/git/etc for grabbing the source you need. Be careful here and don't install too much. We want to maintain an otherwise pristine environment for building our packages. By default changes you make inside the chroot will be wiped. That means those package specific dependencies we'll install each time to build something won't persist.
apt-get install ubuntu-sdk bzr git phablet-tools
exit
By exiting, you'll notice pbuilder will update the base tarball with our changes. Now, when you want to build something, simply do the following:
pbuilder-dist trusty armhf login
bzr branch lp:my-project
apt-get install build-dependencies-you-need
Now, you can build as usual using click tools, so something like
click-buddy --dir .
works as expected. You can even add the --provision to send the resulting click to your device. If you want to grab the resulting click, you'll need to copy it before exiting the chroot, which is mounted on your filesystem under /var/cache/pbuilder/build/. Look for the last line after you issue your login command (pbuilder-dist trusty armhf login). You should see something like,
File extracted to: /var/cache/pbuilder/build//26213
If you cd to this directory on your local machine, you'll see the environment chroot filesystem. Navigate to your source directory and grab a copy of the resulting click. Copy it to a safe place (somewhere outside of the chroot) before exiting the chroot or you will lose your build!
But wait, there's more!
Since you have access to the chroot while it's open (and you can login several times if you wish to create several sessions from the base tarball), you can iteratively build packages as needed, hack on code, etc. The chroot is your playground.
Remember, click is your friend. Happy hacking!
1. Thanks to David Planella for this info
So then what options exist for getting a proper build of a project that will install properly on your device?
A phone can be more than a phone
It can also be a development environment!? Although it's not my recommendation, you can always use the source device to compile the package with. The downsides of this is namely speed and storage space. Nevertheless, it will build a click.
- shell into your device (adb shell / ssh mydevice)
- checkout the code (bzr branch lp:my-project)
- install the needed dependencies and sdk (apt-get install ubuntu-sdk)
- build with click-buddy (click-buddy --dir .)
The click tools contain a handy way to build a chroot expressly suited for use with click-buddy to build things. Basically, we can create a nice fake environment and pretend it's armhf, even though we're not running that architecture.
sudo click chroot -a armhf -f ubuntu-sdk-14.04 create
click-buddy --dir . --arch armhf
Most likely your package will require extra dependencies, which for now will need to be specified and passed in with the --extra-deps argument. These arguments are packages names, just like you would apt-get. Like so;
click-buddy --dir . --arch armhf --extra-deps "libboost-dev:armhf libssl-dev:armhf"
Notice we specified the arch as well, armhf. If we also add a --maint-mode, our extra installed packages will persist. This is handy if you will only ever be building a single project and don't want to constantly update the base chroot with your build dependencies.
Qtcreator build it for me!
Cmake makes all things possible. Qt Creator can not only build the click for you, it can also hold your hand through creating a chroot1. To create a chroot in qtcreator, do the following:
- Open Qt Creator
- Navigate to Tools > Options > Ubuntu > Click
- Click on Create Click Target
- After the click target is finished, add the dependencies needed for building. You can do this by clicking the maintain button.
- Apt-get add what you need or otherwise setup the environment. Once ready, exit the chroot.
- Open qt creator and open the project
- Select armhf when prompted
- You can also manually add the chroot to the project via Projects > Add kit and then select the UbuntuSDK armhf kit.
- Navigate to Projects tab and ensure the UbuntuSDK for armhf kit is selected.
- Build!
So, click can setup a chroot for you, and qt creator can build and manage one too. And these are great options for building one project. However if you find yourself building a plethora of packages or you simply want more control, I recommend setting up and using your own chroot to build. For my own use, I've picked pbuilder, but you can setup the chroot using other tools (like schroot which Qt Creator uses).
sudo apt-get install qemu-user-static ubuntu-dev-tools
pbuilder-dist trusty armhf create
pbuilder-dist trusty armhf login --save-after-login
Then, from inside the chroot shell, install a couple things you will always want available; namely the build tools and bzr/git/etc for grabbing the source you need. Be careful here and don't install too much. We want to maintain an otherwise pristine environment for building our packages. By default changes you make inside the chroot will be wiped. That means those package specific dependencies we'll install each time to build something won't persist.
apt-get install ubuntu-sdk bzr git phablet-tools
exit
By exiting, you'll notice pbuilder will update the base tarball with our changes. Now, when you want to build something, simply do the following:
pbuilder-dist trusty armhf login
bzr branch lp:my-project
apt-get install build-dependencies-you-need
Now, you can build as usual using click tools, so something like
click-buddy --dir .
works as expected. You can even add the --provision to send the resulting click to your device. If you want to grab the resulting click, you'll need to copy it before exiting the chroot, which is mounted on your filesystem under /var/cache/pbuilder/build/. Look for the last line after you issue your login command (pbuilder-dist trusty armhf login). You should see something like,
File extracted to: /var/cache/pbuilder/build//26213
If you cd to this directory on your local machine, you'll see the environment chroot filesystem. Navigate to your source directory and grab a copy of the resulting click. Copy it to a safe place (somewhere outside of the chroot) before exiting the chroot or you will lose your build!
But wait, there's more!
Since you have access to the chroot while it's open (and you can login several times if you wish to create several sessions from the base tarball), you can iteratively build packages as needed, hack on code, etc. The chroot is your playground.
Remember, click is your friend. Happy hacking!
1. Thanks to David Planella for this info
Monday, April 14, 2014
Trusty Release Week: Get your test results in!
As promised, here is your reminder that we are indeed fast approaching the final image for trusty. It's release week, which means it's time to put your energy and focus into finding and getting the remaining bugs documented or fixed in time for the release.
We need you!
The images are a culmination of effort from everyone. I know many have already tested and installed trusty and reported any issues encountered. Thank you! If you haven't yet tested, we need to hear from you!
How to help
The final milestone and images are ready; click here to have a look.
Execute the testcases for ubuntu and your favorite flavor images. Install or upgrade your machine and keep on the lookout for any issues you might find, however small.
I need a guide!
Sound scary? It's simpler than you might think. Checkout the guide and other links at the top of the tracker for help.
I got stuck!
Help is a simple email away, or for real-time help try #ubuntu-quality on freenode. Here are all the ways of getting ahold of the quality team who would love to help you.
Community
Plan to help test and verify the images for trusty and take part in making ubuntu! You'll join a community of people who do there best everyday to ensure ubuntu is an amazing experience. Here's saying thanks, from me and everyone else in the community for your efforts. Happy testing!
We need you!
The images are a culmination of effort from everyone. I know many have already tested and installed trusty and reported any issues encountered. Thank you! If you haven't yet tested, we need to hear from you!
How to help
The final milestone and images are ready; click here to have a look.
Execute the testcases for ubuntu and your favorite flavor images. Install or upgrade your machine and keep on the lookout for any issues you might find, however small.
I need a guide!
Sound scary? It's simpler than you might think. Checkout the guide and other links at the top of the tracker for help.
I got stuck!
Help is a simple email away, or for real-time help try #ubuntu-quality on freenode. Here are all the ways of getting ahold of the quality team who would love to help you.
Community
Plan to help test and verify the images for trusty and take part in making ubuntu! You'll join a community of people who do there best everyday to ensure ubuntu is an amazing experience. Here's saying thanks, from me and everyone else in the community for your efforts. Happy testing!
Monday, March 31, 2014
Time to test trusty!
 |
| Say that three times fast. Time to test trusty, time to test trusty, time to test trusty! |
The developers and early adopters have been working hard all cycle to put forth the best version of ubuntu to date. For you! For all of us! It's time to fix bugs, do last minute polish and prepare for the release candidate which will occur around April 11th.
We need you!
This is were you dear reader come in. You see despite their good looks and wonderful sense of humor and charm, the release team doesn't like to release final images of ubuntu that haven't been thoroughly tested.
 |
| The release team is ready to pounce on untested images |
How to help
Mark your calendars now for April 11th - April 16th. Pick a good date for you and plan to download and test the release candidate image. You'll see a new milestone on the tracker, and an announcement here as well when the image is ready. I won't let you forget, promise!
Execute the testcases for ubuntu and your favorite flavor images. Install or upgrade your machine and keep on the lookout for any issues you might find, however small.
I need a guide!
Sound scary? It's simpler than you might think. Checkout the guide and other links at the top of the tracker for help.
I got stuck!
Help is a simple email away, or for realtime help try #ubuntu-quality on freenode. Here's all the ways of getting ahold of the quality team who would love to help.
Community
Plan to help test and verify the images for trusty and take part in making ubuntu! You'll join a community of people who do there best everyday to ensure ubuntu is an amazing experience. Here's saying thanks, from me and everyone else in the community for your efforts. Happy testing!
Tuesday, March 25, 2014
Trusty looms closer: Final Beta is here!
It may be hard to believe but the next, and dare I say, best, LTS for ubuntu is releasing very soon. We need your help in polishing out critical bugs and issues!
How can I help?
To help test, visit the iso tracker milestone page for 'Trusty Beta 2'. The goal is to
verify the images in preparation for the release. Find those bugs! The information at the top of the page will help you if you need help reporting a bug or understanding how to test.
So what's new?
Besides the usual slew of updates to the applications, stack and kernel, unity has new goodies like minimize on click, menus in toolbar, new lockscreen, and borderless windows!
What if I'm late?
The testing runs through this Thursday March 27th, when the the images for beta 2 will be released. If you miss the deadline we still love getting results! Test against the daily image milestone instead.
Thanks and happy testing everyone!
Labels:
call for testing,
ubuntu
Thursday, March 13, 2014
Keeping ubuntu healthy: Core Apps
Continuing our discussion of testing within ubuntu, today's post will talk about how you can help the community core apps stay healthy.
As you recall the core apps go through a series of QA before being released to the store. However bugs in the application, or in the platform itself can still be exposed. The end result is that the dashboard contains tests failures for that application. To release a new stable image, we need a green dashboard, and more importantly we need to make sure the applications work properly.
Getting plugged in
So to help out, it's important to first plug into the communication stream. After all, we're building these applications and images as a community! First, join the ubuntu phone group on launchpad and sign up for the phone mailing list. The list is active and discussing all issues pertaining to the ubuntu phone. Most importantly, you will see landing team emails that summarize and coordinate issues with the phone images.
From there you can choose a community core app to help improve from a quality perspective. These applications all have development teams and it's helpful to stay in contact with them. Your merge proposal can serve as an introduction!
Finding something to work on
So what needs fixing? A landing team email might point out a failing test. You might notice a test failure on the dashboard yourself. In addition each application keeps a list of bugs reported against it, including bugs that point out failing tests or testing needs. For example here's the list of all new autopilot tests that need to be written for all of the core apps. Pick an app, browse the buglist, assign a bug to yourself, and fix it.
For example, here's the list of bugs for music app. As of this writing you can see several tests that need written, as well as a bug for a test improvement.
You can also simply enhance the app's existing testsuite by fixing a flaky test, or improving the test to use best practices, etc. As a bonus for those reading this near it's original publication date, we just had a session @ vUDS covering the core apps and the testing needs we have. Watch the session / browse the pad and pick something to work on.
Fixing things
Look into any failures you find and have a look at the tests. Often the tests can use a little improvement (or maybe an additional test), and you can help out here! Sometimes failures won't happen every run -- this is the sign of a weird bug, or more likely a flaky test. Fix the test(s), improve them, or add to them. Then commit your work and submit a merge proposal. Follow the guide on the wiki if you need help with doing this.
Remember, you can iteratively run the tests on your device as you work. Read my post on click-buddy for help with this. If you are lacking a device, run the tests on your desktop instead and a reviewer can test against a real device before merging.
Getting Help
For realtime help, check out #ubuntu-quality and #ubuntu-autopilot on freenode. You'll find a group of folks like yourself working on tests, hacking on autopilot and sharing advice. If IRC isn't your thing, feel free to contact us through another method instead. Happy hacking!
As you recall the core apps go through a series of QA before being released to the store. However bugs in the application, or in the platform itself can still be exposed. The end result is that the dashboard contains tests failures for that application. To release a new stable image, we need a green dashboard, and more importantly we need to make sure the applications work properly.Getting plugged in
So to help out, it's important to first plug into the communication stream. After all, we're building these applications and images as a community! First, join the ubuntu phone group on launchpad and sign up for the phone mailing list. The list is active and discussing all issues pertaining to the ubuntu phone. Most importantly, you will see landing team emails that summarize and coordinate issues with the phone images.
From there you can choose a community core app to help improve from a quality perspective. These applications all have development teams and it's helpful to stay in contact with them. Your merge proposal can serve as an introduction!
Finding something to work on
So what needs fixing? A landing team email might point out a failing test. You might notice a test failure on the dashboard yourself. In addition each application keeps a list of bugs reported against it, including bugs that point out failing tests or testing needs. For example here's the list of all new autopilot tests that need to be written for all of the core apps. Pick an app, browse the buglist, assign a bug to yourself, and fix it.
For example, here's the list of bugs for music app. As of this writing you can see several tests that need written, as well as a bug for a test improvement.
You can also simply enhance the app's existing testsuite by fixing a flaky test, or improving the test to use best practices, etc. As a bonus for those reading this near it's original publication date, we just had a session @ vUDS covering the core apps and the testing needs we have. Watch the session / browse the pad and pick something to work on.
Fixing thingsLook into any failures you find and have a look at the tests. Often the tests can use a little improvement (or maybe an additional test), and you can help out here! Sometimes failures won't happen every run -- this is the sign of a weird bug, or more likely a flaky test. Fix the test(s), improve them, or add to them. Then commit your work and submit a merge proposal. Follow the guide on the wiki if you need help with doing this.
Remember, you can iteratively run the tests on your device as you work. Read my post on click-buddy for help with this. If you are lacking a device, run the tests on your desktop instead and a reviewer can test against a real device before merging.
Getting Help
For realtime help, check out #ubuntu-quality and #ubuntu-autopilot on freenode. You'll find a group of folks like yourself working on tests, hacking on autopilot and sharing advice. If IRC isn't your thing, feel free to contact us through another method instead. Happy hacking!
Labels:
ci,
likeimfive,
ubuntu
Thursday, March 6, 2014
Keeping ubuntu healthy: Manual Image Testing
Continuing our discussion of the testing within ubuntu, today's post will talk about how you can help ubuntu stay healthy by manually testing the images produced. No amount of robots or automated testing in the world can replace you (well, at least not yet, heh), and more specifically your workflow and usage patterns.
As discussed, everyday new images are produced for ubuntu for all supported architecture types. I would encourage you to follow along and watch the progression of the OS through these images and your testing. Every data point matters and testing on a regular basis is helpful. So how to get started?
The Desktop
For the desktop images everything you need is on the image tracker. There is a wonderful video and text tutorial for helping you get started. You report your results on the tracker itself in a simple web form, so you'll need a launchpad account if you don't have one.
The secondary way to help keep the desktop images in good shape is to install and run the development version of ubuntu on your machine. Each day you can check for updates and update your machine to stay in sync. Use your pc as you normally would, and when you find a bug, report it! Bugs found before the release are much easier to fix than after release.
Phablet
Now for the phablet images you will need a device capable of running the image. Check out the list. Grab your device and go through the installation process as described on the wiki. Make sure to select the '-proposed' channel when you install so you will see updates to get the latest images being worked on every time they are built. From there you can update everyday. Use the device and when you find a bug, report it! Here's a wiki page to help guide your testing and help you understand how and where to report bugs.
Don't forget there's a whole team of people within ubuntu dedicated to testing just like you. And they would love to have you join them!
As discussed, everyday new images are produced for ubuntu for all supported architecture types. I would encourage you to follow along and watch the progression of the OS through these images and your testing. Every data point matters and testing on a regular basis is helpful. So how to get started?
 |
| Settle in with a nice cup of tea while testing! |
The Desktop
For the desktop images everything you need is on the image tracker. There is a wonderful video and text tutorial for helping you get started. You report your results on the tracker itself in a simple web form, so you'll need a launchpad account if you don't have one.
The secondary way to help keep the desktop images in good shape is to install and run the development version of ubuntu on your machine. Each day you can check for updates and update your machine to stay in sync. Use your pc as you normally would, and when you find a bug, report it! Bugs found before the release are much easier to fix than after release.
Phablet
Now for the phablet images you will need a device capable of running the image. Check out the list. Grab your device and go through the installation process as described on the wiki. Make sure to select the '-proposed' channel when you install so you will see updates to get the latest images being worked on every time they are built. From there you can update everyday. Use the device and when you find a bug, report it! Here's a wiki page to help guide your testing and help you understand how and where to report bugs.
Don't forget there's a whole team of people within ubuntu dedicated to testing just like you. And they would love to have you join them!
Labels:
likeimfive,
testing,
ubuntu
Monday, March 3, 2014
A simple look at testing within ubuntu
Since just before the last LTS, quality has been a buzzword within the ubuntu community. We've come a long way since precise and I wanted to provide some help and prospective on what ubuntu's process for quality looks like this cycle. In simple terms. Or as reddit would say, "explain to me like I'm 5".
I'll try and define terms as we go. First let me define CI, which is perhaps the buzzword of this cycle, lest I lose all of you! CI stands for continuous integration, and it means we are testing ubuntu. All the time. Non-stop. Every change we make, we test. The goal behind this idea is to find and fix problems, before well, they become problems on your device!
CI Dashboard
The CI dashboard then is a way to visually see the results of this testing. It acts as a representation of the health of ubuntu as a distribution. At least once a day runs are executed, apps and images are tested and benchmarked, and the results are populated on ci.ubuntu.com. This is perhaps the most visible part of the CI (our continuous testing efforts) that is happening within ubuntu. But let's step back a minute and look at how the overall CI process works within ubuntu.
CI Process
App developers hack on a bit of code, fixing bugs or adding new features to the codebase. Once the code is ready, a merge proposal1 is created by the developer and feedback is sought. If the code passes the peer review and the application's tests, it will then become part of the image after a journey through the CI train.
For the community core apps, the code is merged after peer review, and then undergoes a similar journey to the store where it will become part of the image as well. Provided of course it meets further review criteria by myself and Alan (we'll just call him the gatekeeper).
Lest we forget, upstream2 uploads3 are done as well. We can hope some form of testing was done on them before we received them. Nevertheless, tests are run on these as well, and if they pass successfully, the new packages will enter the archive4 and become part of the image.
Generating Images
Now it's time to generate some images. For the desktop a snapshot of what's in the ubuntu archive is taken each day, built, and then subjected to a series of installation tests. If the tests pass, it is released for general testing called Image (or ISO) testing. An image is tested and declared stable as part of a milestone (or testing event) and can become the next version of ubuntu!
On the ubuntu phone side of things, all the new uploads are gathered and evaluated for risk. If something is a large change, it might be prudent to not land it with other large changes so we can tell what broke should the image not work properly. Once everything is ready, a new image is created and is released for testing. The OTA updates (over-the-air; system updates) on your ubuntu phone come from this process!
How you can help?
Still with me I hope? As you can see there's many things happening each day in regards to quality and lots of places where you can create a positive change for the health of the distro! In my next few posts, I'll cover each of the places you can plug in to help ubuntu be healthy everyday!
1. A merge proposal is a means of changing an applications code via peer review.
2. By upstream, I mean the communities and people who make things we use inside of ubuntu, but are not directly a part of it. Something like the browser (firefox) and kernel are good examples.
3. This can happen via a general sync at the beginning of the cycle from debian. This sync copies part of the debian archive into the ubuntu archive, which in effect causes applications to be updated. Applications are also updated whenever a core ubuntu developer or a MOTU uploads a new version to the archive.
4. In case you are wondering, "the archive" is the big repository where all of your updates and new applications come from!
I'll try and define terms as we go. First let me define CI, which is perhaps the buzzword of this cycle, lest I lose all of you! CI stands for continuous integration, and it means we are testing ubuntu. All the time. Non-stop. Every change we make, we test. The goal behind this idea is to find and fix problems, before well, they become problems on your device!
CI Dashboard
The CI dashboard then is a way to visually see the results of this testing. It acts as a representation of the health of ubuntu as a distribution. At least once a day runs are executed, apps and images are tested and benchmarked, and the results are populated on ci.ubuntu.com. This is perhaps the most visible part of the CI (our continuous testing efforts) that is happening within ubuntu. But let's step back a minute and look at how the overall CI process works within ubuntu.
CI Process
App developers hack on a bit of code, fixing bugs or adding new features to the codebase. Once the code is ready, a merge proposal1 is created by the developer and feedback is sought. If the code passes the peer review and the application's tests, it will then become part of the image after a journey through the CI train.
For the community core apps, the code is merged after peer review, and then undergoes a similar journey to the store where it will become part of the image as well. Provided of course it meets further review criteria by myself and Alan (we'll just call him the gatekeeper).
 |
| Though menacing, Alan assures me he doesn't bite |
Lest we forget, upstream2 uploads3 are done as well. We can hope some form of testing was done on them before we received them. Nevertheless, tests are run on these as well, and if they pass successfully, the new packages will enter the archive4 and become part of the image.
Generating Images
Now it's time to generate some images. For the desktop a snapshot of what's in the ubuntu archive is taken each day, built, and then subjected to a series of installation tests. If the tests pass, it is released for general testing called Image (or ISO) testing. An image is tested and declared stable as part of a milestone (or testing event) and can become the next version of ubuntu!
 |
| Adopted images are healthy images! |
On the ubuntu phone side of things, all the new uploads are gathered and evaluated for risk. If something is a large change, it might be prudent to not land it with other large changes so we can tell what broke should the image not work properly. Once everything is ready, a new image is created and is released for testing. The OTA updates (over-the-air; system updates) on your ubuntu phone come from this process!
How you can help?
Still with me I hope? As you can see there's many things happening each day in regards to quality and lots of places where you can create a positive change for the health of the distro! In my next few posts, I'll cover each of the places you can plug in to help ubuntu be healthy everyday!
1. A merge proposal is a means of changing an applications code via peer review.
2. By upstream, I mean the communities and people who make things we use inside of ubuntu, but are not directly a part of it. Something like the browser (firefox) and kernel are good examples.
3. This can happen via a general sync at the beginning of the cycle from debian. This sync copies part of the debian archive into the ubuntu archive, which in effect causes applications to be updated. Applications are also updated whenever a core ubuntu developer or a MOTU uploads a new version to the archive.
4. In case you are wondering, "the archive" is the big repository where all of your updates and new applications come from!
Labels:
ci,
likeimfive,
ubuntu
Thursday, February 20, 2014
Click-buddy and you: how to test your ubuntu sdk application
Recently the ubuntu core app developers and myself have been on an adventure towards adopting cmake for the all the core applications. While some of the applications are pure qml it's still been useful to embark on adopting a singular build system for all of the projects. Now that (most) of the pain of transitioning is gone, I'm going to talk about one of the useful features of setting up cmake for your project; click-buddy!
Click-buddy is an evolving tool that helps you build and deploy click packages to your phablet device. In addition, it has the ability to setup the device to run your autopilot test suite. So, rather than writing anything further, let's cover an example. You are going to need phablet-tools installed for this to work. I'm going to branch the clock app, build the click package, install it on my device, and finally run the tests.
bzr branch lp:ubuntu-clock-app
cd ubuntu-clock-app
click-buddy --dir . --provision
phablet-test-run -v ubuntu_clock_app
Click-buddy is also gaining the ability to build your project, even it involves a plugin and you are interested in building for your phablet device (armhf). Once landed you will be able to run something like this for non-qml applications.
sudo click chroot -a armhf create
click-buddy --arch armhf --provision
This will setup a chroot automagically for you and compile and build your application. Give it a try!
Note, as of this writing, emulator support for the ubuntu-ui-toolkit emulator is not yet built in. If your tests fail with a module import, run this line from your connected pc. It will copy over the ubuntu-ui-toolkit emulator (provided you have it installed on your pc :-) ) and your tests should now properly run.
adb push /usr/lib/python2.7/dist-packages/ubuntuuitoolkit /home/phablet/autopilot/ubuntuuitoolkit
Click-buddy is an evolving tool that helps you build and deploy click packages to your phablet device. In addition, it has the ability to setup the device to run your autopilot test suite. So, rather than writing anything further, let's cover an example. You are going to need phablet-tools installed for this to work. I'm going to branch the clock app, build the click package, install it on my device, and finally run the tests.
bzr branch lp:ubuntu-clock-app
cd ubuntu-clock-app
click-buddy --dir . --provision
phablet-test-run -v ubuntu_clock_app
Click-buddy is also gaining the ability to build your project, even it involves a plugin and you are interested in building for your phablet device (armhf). Once landed you will be able to run something like this for non-qml applications.
sudo click chroot -a armhf create
click-buddy --arch armhf --provision
This will setup a chroot automagically for you and compile and build your application. Give it a try!
Note, as of this writing, emulator support for the ubuntu-ui-toolkit emulator is not yet built in. If your tests fail with a module import, run this line from your connected pc. It will copy over the ubuntu-ui-toolkit emulator (provided you have it installed on your pc :-) ) and your tests should now properly run.
adb push /usr/lib/python2.7/dist-packages/ubuntuuitoolkit /home/phablet/autopilot/ubuntuuitoolkit
Subscribe to:
Posts (Atom)