Yocto part III – a custom meta layer

In this part of the series (previous: i, ii) we will have a look at setting up our own meta layer, so that we can make changes without having to fork the Yocto recipes.

The easiest way to create a layer is to use the yocto-layer tool. To fully understand what a layer consists of, I recommend looking at the instructions for creating a layer manually as well. So, long story short:

cd ..
yocto-layer create diet

The name, meta-diet, comes from my goal with this series – to create a minimal fast booting image. Only time will tell how well that goes.

The yocto-layer create command produces an MIT-licensed layer. The MIT license is a weak copyleft, so you can choose to close it. I’ll leave it open for your enjoyment. Also, remember to change the README file. It contains a few references to xxxx and yyyy, which would be you and your project.

As we don’t want the layer to be a part of the layers provided by the Yocto project, we need to separate it out. To do this, change the directory so that you stand in the meta-diet directory and run:

git init

Now you are free to do what you like. I suggest making a commit of what is there and perhaps add a remote so that you can push your changes somewhere.

An empty layer is only so much fun, so lets start by adding a recipe for your core-image-minimal, enhanced with systemd-analyze. First, let’s create a directory for the recipe:

mkdir -p recipes-diet/images

In that directory, create the diet-image.bb file using your favorite, non-emacs, editor. Put the following code in it. This means that we use the core-image-minimal as base, and add systemd-analyze to what we want to install onto the image.

require recipes-core/images/core-image-minimal.bb

IMAGE_INSTALL += "systemd-analyze"

As copying code from a blog post is boring, I’ve put the layer on github as meta-diet. For this installment, use the part-iii tag.

To build the diet image, change directory to the build directory and edit the conf/bblayers.conf file. Simply add the new layer to the BBLAYERS variable. Now you can bitbake the new image as simple as:

bitbake diet-image

So, this installment got us nowhere feature wise, but now we have everything in place to start experimenting!

Yocto part II – baseline image size

In the first installment of this series, we established a base line image and had a look at the initial boot performance. This time, we will establish a baseline for the image size. In an embedded system, these two factors often go hand in hand, as reading data from FLASH often consumes a considerable amount of time during boot.

It is fairly easy to see how large the resulting root file system and kernel are, but we need more detail. To enable some statistics for the sizes of the various packages built, we first need to enable build history and rebuild our baseline image. Build history is enabled by adding the following lines to the local.conf file.

INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"

This produces a set of files with statistics in buildhistory/images/qemux86/eglibc/core-image-minimal. The files-in-image.txt file, a detailed list of all the files of the system, along with their sizes, is listed. In the installed-package-sizes.txt, we get a list of responsible packages. The list for a baseline image can be seen below.

8641 KiB systemd
4487 KiB udev-hwdb
3042 KiB libc6
1290 KiB libx11-6
1221 KiB e2fsprogs-e2fsck
1153 KiB shadow
798  KiB dbus-1
547  KiB kmod
532  KiB busybox
398  KiB udev
350  KiB libkmod2
299  KiB libdbus-1-3
297  KiB libext2fs2
261  KiB libmount1
249  KiB udev-utils
241  KiB libblkid1
233  KiB systemd-analyze
161  KiB liblzma5
159  KiB libexpat1
133  KiB libxcb1
110  KiB v86d
94   KiB util-linux-fsck
86   KiB libz1
83   KiB libgcc1
36   KiB systemd-binfmt
32   KiB kernel-module-uvesafb
31   KiB util-linux-mount
31   KiB libwrap0
30   KiB libacl1
29   KiB util-linux-agetty
27   KiB libe2p2
23   KiB netbase
20   KiB kernel-3.14.0-yocto-standard
19   KiB libxdmcp6
16   KiB libattr1
15   KiB libcap2
14   KiB libuuid1
11   KiB kernel-module-binfmt-misc
10   KiB libcom-err2
9    KiB libxau6
5    KiB update-rc.d
4    KiB update-alternatives-opkg
4    KiB base-files
3    KiB busybox-udhcpc
2    KiB shadow-securetty
2    KiB run-postinsts
1    KiB systemd-serialgetty
1    KiB busybox-syslog
0    KiB systemd-compat-units
0    KiB packagegroup-core-boot
0    KiB base-passwd

In addition to this, the bz kernel image is 6.2MiB.

My approach to this system will be that of a gadget maker, so when trying to optimize the size of the system, I will try to use as much as possible of what I know of the target hardware to minimize the size. This ought to make it possible to reduce the kernel, the udev-hwdb and more. In addition to this, Lennart says that systemd is tweakable when it comes to size. I’m looking forward to trying this.

That is all for this installment. Next time we will have a look at setting up our own meta layer to start playing with, then we will try to create a truly minimalist and fast booting image.

Yocto part I – baseline boot time

The Yocto project provides a set of tools to build custom distribution images from scratch. When using Yocto, the image, and all the tooling used to build the image, is built from recipes. These recipes are parsed using the bitbake command. The recipes have dependencies, just as ordinary packages in a classical distro. By pointing to an image recipe, a dependency tree will be constructed and a large number of packages will be downloaded, built and then assembled into a single image.

In this little series of blog posts, I aim to have a look at what can be done about size and performance for the minimal core image that comes with the system. The target system will be an x86 QEMU machine. So, my starting point is the outcome of the Yocto Project Quick Start. Before I get started, let me also point out the excellent Yocto Project Reference Manual.

My starting point is the core-image-minimal recipe, creating a minimal Linux system. It uses busybox, and provides very little, so it is a good baseline. I’d like to use systemd for booting the image, so before building, I added the following lines to my local.conf file (info source).


DISTRO_FEATURES_append = " systemd"
VIRTUAL-RUNTIME_init_manager = "systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_initscripts = ""

I’m also keen on having a look at the boot performance, so I added systemd-analyze to the IMAGE_INSTALL variable. I did this by modifying the meta/recipes-core/images/core-image-minimal.bb file. I know that this is bad practice, I will explain how to do this properly using a custom meta layer and image recipe in a later installment of this series.

Having built and started the system, I get a boot time of 11.3s. Of these, 6.4s are spent to start the kernel, while the remaining 4.9s are spent in userspace. I also got a nice boot graph, as well as a list of systemd units to blame for the time the boot sequence consumed. I used a little netcat trick to get these files of the QEMU system to my host machine. Always nice, to avoid adding cruft to the image.


1.272s systemd-udev-trigger.service
991ms systemd-user-sessions.service
694ms systemd-logind.service
363ms systemd-networkd.service
360ms systemd-remount-fs.service
353ms systemd-sysctl.service
299ms sys-kernel-debug.mount
292ms kmod-static-nodes.service
269ms dev-mqueue.mount
250ms systemd-tmpfiles-setup.service
242ms systemd-journal-flush.service
137ms systemd-update-utmp.service
129ms systemd-tmpfiles-setup-dev.service
102ms systemd-udevd.service
91ms tmp.mount
78ms systemd-random-seed.service
64ms var-volatile.mount

 

Boot Chart
The boot chart.

Next time, we will have a closer look on the base line image size, before we start looking at what we can do to optimize things.

Qt Contributors Summit 2014

I’m writing from a warm and sunny Berlin, having attended my first Qt Contributors Summit. It is always nice to meet the Qt developer team, and to get a chance to hang out with the Pelagicorians from our Munich office.

QtCS2014This conference really acts as a good balance to Qt Developer Days. Where QtDD consists of speeches and focuses on information for the users of Qt, the QtCS consists of open discussions focusing on the development of Qt. I, personally, will take two things with me from here: the vibrant community and the openness of Qt.

It is nice to see the community in action. There are people from KDE, Digia, KDAB, ICS, Intel, Canonical, Jolla, Pelagicore and more attending.  This really shows how strong Qt has grown through the qt-project. Opening up development to people outside what was Trolltech / Nokia really has resulted in a strong community.

Having people from potentially competing companies at a joint developer conference can be a difficult thing. Here, however, the discussions have been open and honest. As a developer, it is really nice to see that Qt is driven by developers and not by politics. The discussions have been really great, letting everyone discuss their concerns and use-cases and agreeing on the direction going  forward. All of you can read the minutes at the QtCS wiki.

Finally, I’d like to thank Pelagicore for letting me attend, and for sponsoring the QtCS this year.

Another hacknight in Gothenburg

The following announcement is in Swedish. Please ignore it if it does not make sense to you!

foss-gbgVälkomna på foss-gbg hackafton!

Den 28/5 17:00 träffas vi för att lära oss om GNOME.

Andreas Nilsson och Mattias Bengtsson introducerar projektet, verktygen och hjälper er komma igång.

Vid åttasnåret drar vi vidare och umgås över en öl.

Pelagicore står för lokaler och bjuder på lättare tilltugg under tillställningen.

Inget inträde, men begränsat antal platser. Anmälan på eventbrite.

Welcome to another hack-night in Gothenburg

This post is in Swedish, please ignore if it does not make sense to you.

Välkomna till en hackafton den 23/4.

Vi träffas och hackar på våra projekt, samt lyssnar till två korta föredrag. Vi bjuder på lättare förtäring under hackandet och sedan
drar vi vidare till Biljardpalatset och umgås.

Spelar licensen någon roll?

Henrik o Rikard / Morus
Vi tittar på hur företag idag använder FOSS. Vilka möjligheter finns och vilka trender ser vi nu. Vi börjar och avslutar med ett licens-Quiz.

Introduktion till BeagleBone Black

Johan o Jeremiah / Pelagicore
Vi presenterar BeagleBone Black-plattformen – en perfekt utgångspunkt för Linux-baserade hårdvaruhack. Vi bygger en Yocto-baserad distro för den, men kör även en färdigbyggd Debian-distro.

Pelagicore sponsrar med mat och lokaler!

Qt5 Cadaques 2014-01 released

We are happy to announce the 2014-01 release of Qt 5 Cadaques, a book on Qt. We’ve worked away over the holidays and are happy to bring you chapter 11, on networking. We’ve also restructured chapters 15 and 16, dealing with C++ and native code, so that they are ready to be fleshed out for our next release. In the existing text, we’ve made lots of updates to fix typos and grammar issues.

In addition to fixing and adding contents, we’ve worked on the site. We’ve changed the theme to “flatly”, as we feel that this makes the site nicer to look at and to read from. We’ve also started experimenting with building a PDF/ePUB release. You can find the download links for these at the top of the table of contents. We know that both these formats need polishing, but we will focus our efforts on completing the contents first.

Before I stop and let you all start reading I would like to thank everyone who has visited the site and read the book. The response has been fantastic and we’ve seen so many positive comments, received so many great issue reports and seen the traffic to this site really grow. Thank you for reading!

One week of Jolla

So I’ve enjoyed my first few days as a Jolla user. Before I start discussing Jolla and SailFishOS, something about my phone background. I’ve come from Symbian S60, to the N900, then N9 followed by various Androids before getting my Jolla just before Christmas (great timing – thanks!). This means that I came from a non-service oriented background to Android, and now I finally realize the strength of Android’s service lock-in first-hand. It is really inconvenient moving from Android and Google’s official clients for their services to a platform with third party clients to existing services.

My feelings regarding Jolla are mixed. I want them to succeed. I believe that they are right providing a platform based on Qt, native code and a more generic Linux stack and Android. Unfortunately, I think that this is a classic second system. All ideas have been taken a bit to far, and the end result is… not ideal.

When the N9 arrived it introduced swipes on a whole new level. The level was just right. Everything felt natural, and going to a Samsung S3 felt really clumsy. Since then, swipes have been established by most platforms. On Android you swipe from the top to reach the settings and notifications, in GMail you archive mails by swiping sideways, etc. On the Jolla, there are too many layers of swipeable surfaces, and the simpleness breaks as the user cannot predict what will happen. An example is the photo gallery. Tap to enters the camera photos, tap a photo to enter that photo, and now try to get back out. Swiping sideways moves you between the photos, so swiping back does not work. Instead, you have to tap the photo to get a header with sharing options. Now you can swipe back in that header. Not very intuitive.

Another interesting swipe experience is in the mail client when composing a mail. Then you can reach the top menu by swiping down, or up. Not really a problem, but confusing.

One of the parts of the platform that I was curious about was the Android integration. The Jolla comes with Yandex store. I added the HumbleBundle Android app as well. Compared to my Android devices, the experience is really good. Installations are quick, app performance is good. The only “glitch” is that the Android instance is treated as one app inside SailFishOS, so it is not possible to directly switch between Android apps without first picking the last used Android app and then swithing inside Android.

During my time as a Jolla user I’ve seen two software updates, so the platform is evolving. Right now it feels like a Nokia device. The hardware looks really great and really provides a good performance. Unfortunately, the software is lacking. I hope that the Jolla sailors will have the stamina and strength to take the platform all the way. I really want them to succeed.

Qt 5 Cadaques is Released!

Juergen Bocklage-Ryannel and myself are writing a book! As we’ve reached the point where we want to share our work with the workd, we are happy to open the doors to the book Qt 5 Cadaques – A Book about Qt 5! It all started a couple of years back, when two developers saw a need: there where no books approaching Qt from the QML side. We decided to solve the problem by writing a book.

During these years, lots has happened in the land of Qt. The ecosystem around the Qt Project has stabilized after the whole Microsoft-Nokia thing. Jolla has released a phone with Qt 5, as has Blackberry. Ubuntu Phone will be based on Qt 5. The Raspberry Pi has taken on the whole world with a great hardware that runs great with Qt 5. It is now possible to target both Android and iOS using Qt 5.

Looking at these changes, one can see that Qt really has grown towards mobile. Towards device user interfaces. User interfaces where the look and feel has to be integrated into the context of a physical device. This is where Qt Quick and QML really shines. Before I go on, it is worth pointing out that the Qt ecosystem has grown towards mobile, it has not moved there. The cross platform desktop support is still there and works just as great as it has for almost 20 years now!

For aspiring Qt developers, there is a plethora of documentation for using Qt from C++. There are multiple books on the topic. However, when targetting devices, C++ is only a part of the story. Here, Qt Quick and QML are equally important. That is why we are happy to make Qt 5 Cadaques available!

A final word of warning. The book is being written as you read it. We believe in the open source mantra release early, release often. This means that the text currently is in its early stages. We have setup a feedback system so that you as a reader can help us improve the material by pointing out issues. We are also looking for ways to grow the contents. We also keep a project blog on the site to announce our releases and to ask you about your opinion on the book. We would love for you to tell us what you want to read about!