Tuesday, December 2, 2014

Creating multi-arch click packages

Click packages are one of the pieces of new technology that drives the next version of ubuntu on the phone and desktop. In a nutshell click packages allow for application developers to easily package and deliver application updates independent of the distribution release or archive. Without going into the interesting technical merits and de-merits of click packages, this means the consumer can get faster application updates. But much of the discussion and usage of click packages until now has revolved around mobile. I wanted to talk about using click packages on the desktop and packaging clicks for multiple architectures.

The manifest file
Click packages follow a specific format. Click packages contain a payload of an application's libraries, code, artwork and resources, along with its needed external dependencies. The description of the package is found in the manifest file, which is what I'd like to talk about. The file must contain a few keys, but one of the recognized optional keys is architecture. This key allows specifying architectures the package will run on.

If an application contains no compiled code, simply use 'all' as the value for architecture. This accomplishes the goal of running on all supported architectures and many of the applications currently in the ubuntu touch store fall into this category. However, an increasing number of applications do contain compiled code. Here's how to enable support across architectures for projects with compiled code.

Fat packages
The click format along with the ubuntu touch store fully support specifying one or more values for specific architecture support inside the application manifest file. Those values follow the same format as dpkg architecture names. Now in theory if a project containing compiled code lists the architectures to support, click build should be able to build one package for all. However, for now this process requires a little manual intervention. So lets talk about building a fat (or big boned!) package that contains support for multiple architectures inside a single click package.

Those who just want to skip ahead can check out the example package I put together using clock. This same package can be found in the store as multi-arch clock test. Feel free to install the click package on the desktop, the i386 emulator and an armhf device.

Building a click for a different architecture
To make a multi-arch package a click package needs to be built for each desired architecture. Follow this tutorial on developer.ubuntu.com for more information on how to create a click target for each architecture. Once all the targets are setup, use the ubuntu sdk to build a click for each target. The end result is a click file specific to each architecture.

For example in creating the clock package above, I built a click for amd64, i386 and armhf. Three files were generated:

com.ubuntu.clock_3.2.176_amd64.click
com.ubuntu.clock_3.2.176_i386.click
com.ubuntu.clock_3.2.176_armhf.click

Notice the handy naming scheme allows for easy differentiation as to which click belongs to which architecture. Next, extract the compiled code from each click package. This can be accomplished by utilizing dpkg. For example,

dpkg -x com.ubuntu.clock_3.2.176_amd64.click amd64

Do this for each package. The result should be a folder corresponding to each package architecture.

Next copy one version of the package for use as the base of multi-arch click package. In addition, remove all the compiled code under the lib folder. This folder will be populated with the extracted compiled code from the architecture specific click packages.

cp amd64 multi
rm -rf multi/lib/*

Now there is a folder for each click package, and a new folder named multi that contains the application, minus any compiled code.

Creating the multi-arch click
Inside the extracted click packages is a lib folder. The compiled modules should be arranged inside, potentially inside an architecture subfolder (depending on how the package is built).

Copy all of the compiled modules into a new folder inside the lib folder of the multi directory. The folder name should correspond to the architecture of the complied code. Here's a list of the architectures for ARM, i386, and amd64 respectively.


arm-linux-gnueabihf
i386-linux-gnu
x86_64-linux-gnu


You can check the naming from an intended device by looking in the application-click.conf file.

grep ARCH /usr/share/upstart/sessions/application-click.conf

To use the clock package as an example again, here's a quick look at the folder structure:

lib/arm-linux-gnueabihf/...
lib/i386-linux-gnu/...
lib/x86_64-linux-gnu/...

The contents of lib/* from each click package I built earlier is under a corresponding folder inside the multi/lib directory. So for example, the lib folder from com.ubuntu.clock_3.2.176_i386.click became lib/i386-linux-gnu/.

Presto, magic package time! 
Finally the manifest.json file needs to be updated to reflect support for the desired architectures. Inside the manifest.json file under the multi directory, edit the architecture key values to list all supported architectures for the new package. For example to list support for ARM and x86 architectures,

"architecture": ["armhf", "i386", "amd64"],

To build the new package, execute click build multi. The resulting click should build and be named with a _multi.click prefix. This click can be installed on any of the specified architectures and is ready to be uploaded to the store.

Caveats, nibbly bits and bugs
So apart from click not automagically building these packages, there is one other bug as of this writing. The resulting multi-arch click will fail the automated store review and instead enter manual review. To workaround this request a manual review. Upon approval, the application will enter the store as usual.

Summary
In summary to create a multi-arch click package build a click for each supported architecture. Then pull the compiled library code from each click and place into a single click package. Next modify the click manifest file to state all of the architectures supported. Finally, rebuild the click package!

I trust this explanation and example provides encouragement to include support for x86 platforms when creating and uploading a click package to the store. Undoubtedly there are other ways to build a multi-arch click; simply ensure all the compiled code for each architecture is included inside the click package. Feel free to experiment!

If you have any questions as usual feel free to contact me. I look forward to seeing more applications in the store from my unity8 desktop!

1 comment: