Posts filed under ‘Qt’

icecc setup how to

I wrote this article back last February just to document the way I did setup icecc in my home and decided to put it here for easy access. Just ignore the time of compilation for both Qt and webkit (since then, both have changed considerably and I also added another node into the cluster, an Intel i7). For reference, the trick of  combining ccache and icecc is not very well know and this is why I think it is still worthwhile to document it here.

Maybe those are techniques that you are already familiar with, so my advanced apologies. If you don’t have used icecc/ccache before or just want to known a little more about it, let’s proceed.

Ok, so this is the scenario: you are working with a big code base or maybe you need to recompile a specific version of Qt/WebKit. Or it can be another reason, maybe you just want to self compile your own kernel/desktop like real programmers do, right?

Even with today’s multicore/gigabyte based systems, it may take a little longer than you wish to compile it all: http://xkcd.com/303/

So, what to do in this cases? There are some projects that can help you out, let’s examine the case of ccache first.

Ccache (http://ccache.samba.org/) is a tool created by samba team, that project that offers CIFS/SMB protocol implementation over *nix kernels. It works with a simple principle of generating a cache of compiled files and reusing those object files if no changes in the source code were detected. Instalation is quite easy in Linux (sudo apt-get install ccache), and you only need to point your PATH environment variable to ccache.

An alternative is to have a file in your home (e.g. ~/.samba.bashrc) with the following function (and source it in .bashrc):
function sambacache {
export PATH=/usr/lib/ccache:$PATH
}

So that you can enable samba by doing:
adenilson@macmini:~$ which gcc
/usr/bin/gcc
adenilson@macmini:~$ sambacache
adenilson@macmini:~$ which gcc
/usr/lib/ccache/gcc

The first compilation will take the same time as you are used to, but the following ones (after the cache is populated i.e. check ~/.ccache directory) will give huge speed ups. So, as an example, a favorite of mine (http://cellardoor.googlecode.com) takes 4s to compile in a Intel i5@2.3Ghz the fist time but next it will take only 0.48s to recompile after a make clean. Pretty neat, hum?

Ccache adds almost no overhead in the compilation and is a perfect helper if you have just 1 computer to do your compilation jobs.

There is an issue with the way that ccache works. It is easy to see that if your project has many self-generated files by each new compilation, the cache will not be reused. So, if for recompiling just Qt there are not big benefits (it self generates lots of header files at beginning of compilation), for WebKit it brings the time compilation down to just 2 minutes.

The next project is icecc (http://en.opensuse.org/Icecream) and it allows to use several computers in the same network to distribute the compilation jobs. It is the ‘evolution’ of distcc, offers some extra features and has a great monitoring tool called ‘icemon’. Instalation is easy in Linux (sudo apt-get install icecc icecc-monitor), configuration not that easy and the documentation could be better.

The speed ups are almost linear, being the network speed the most probable bottleneck. I’ve used it in my previous job with up to 15 computers connected and it allowed me to do a cold recompile of Qt in less than 8 minutes (bear in mind that those 15 computers were also compiling other projects at same time…).

To make it easier to follow, I will explain my home setup. I have 2 machines:
a) macmini: Intel i5@2.3Ghz running Linux 32 bits (dual boot with OSX Lion).

b) blackbloat: Acer notebook (cheapest one I could find, so portability is not its strongest feature…) Intel i5@2.67Ghz running linux 64 bits (dual boot with Windows 7). IP is 192.168.1.135

What is neat about icecc is the fact that you can have varied nodes in your cluster and still perform compilation jobs distribution. It does that by allowing you to create a ‘rootstrap’ of libraries and compilers to be used, being those sent to the other nodes in your first compilation. To do it, you should run:

adenilson@blackbloat:~$ icecc –build-native
adding file /usr/bin/gcc
adding file /lib/x86_64-linux-gnu/libc.so.6
adding file /lib64/ld-linux-x86-64.so.2
adding file /usr/bin/g++
adding file /usr/bin/as
adding file /usr/lib/libopcodes-2.21.53-system.20110810.so
adding file /usr/lib/libbfd-2.21.53-system.20110810.so
adding file /lib/x86_64-linux-gnu/libz.so.1
adding file /lib/x86_64-linux-gnu/libdl.so.2
adding file /usr/bin/cc1=/usr/lib/gcc/x86_64-linux-gnu/4.6.1/cc1
adding file /usr/lib/libmpc.so.2
adding file /usr/lib/libmpfr.so.4
adding file /usr/lib/libgmp.so.10
adding file /usr/bin/cc1plus=/usr/lib/gcc/x86_64-linux-gnu/4.6.1/cc1plus
adding file /usr/lib/gcc/x86_64-linux-gnu/4.6.1/liblto_plugin.so
adding file /etc/ld.so.conf=/tmp/icecc_ld_so_confpM2h21
creating de07a31507267d47693646853b78125e.tar.gz

adenilson@macmini:~$ icecc –build-native
adding file /usr/bin/gcc
adding file /lib/i386-linux-gnu/libc.so.6
adding file /lib/ld-linux.so.2
adding file /usr/bin/g++
adding file /usr/bin/as
adding file /usr/lib/libopcodes-2.21.53-system.20110810.so
adding file /usr/lib/libbfd-2.21.53-system.20110810.so
adding file /lib/i386-linux-gnu/libz.so.1
adding file /lib/i386-linux-gnu/libdl.so.2
adding file /usr/bin/cc1=/usr/lib/gcc/i686-linux-gnu/4.6.1/cc1
adding file /usr/lib/libmpc.so.2
adding file /usr/lib/libmpfr.so.4
adding file /usr/lib/libgmp.so.10
adding file /usr/bin/cc1plus=/usr/lib/gcc/i686-linux-gnu/4.6.1/cc1plus
adding file /usr/lib/gcc/i686-linux-gnu/4.6.1/liblto_plugin.so
adding file /etc/ld.so.conf=/tmp/icecc_ld_so_confWHgW78
creating 73c127da41faa91bfdbae1faedbb2113.tar.gz

And later point the environment variable ICECC_VERSION to where is the rootstrap tarball. In my case, I added a file in my home (i.e. .icecc.bashrc) and renamed the tarball with the following:

function coolice {
export ICECC_VERSION=/home/adenilson/Desktop/ice_rootstrap.tar.gz
export PATH=/usr/lib/icecc/bin/:$PATH
}

Similar to ccache, icecc requires to add it in the PATH before the standard compiler.
adenilson@macmini:~$ which gcc
/usr/bin/gcc
adenilson@macmini:~$ coolice
adenilson@macmini:~$ which gcc
/usr/lib/icecc/bin//gcc

The final step is to choose which machine will run the scheduler, responsible for distributing the compiler jobs between the nodes. In my case, I picked blackbloat, for making the changes permanent, added in /etc/default/icecc
START_ICECC=”true”
START_ICECC_SCHEDULER=”true”

And all the nodes (including blackbloat, since I doubled it as both a scheduler and a compiler node) should know which network to connect to (I decided to call my network ‘tucks’) thus editing the file /etc/icecc/icecc.conf:

ICECC_NETNAME=”tucks”
ICECC_ALLOW_REMOTE=”yes”
ICECC_SCHEDULER_HOST=”192.168.1.135″

There are other parameters you can tweak. For example, by default icecc will assign a compiler job for each CPU/Core/virtual cpu in your system, so it would run at maximum 4 jobs in each node in the case of an intel i5 (2 cores + hyper thread). You can change that by assigning a value in ICECC_MAX_JOBS filed in icecc.conf. Another tweak is the nice level of each running job process, default is 5 (which is good for not disturbing the normal workflow in a computer, but you can get a slightly better performance by changing it to a lesser value).

To monitor the machines in the cluster, just use: icemon -n tucks (see attached image).

It is important to known if *really* the jobs are being distributed. You can check:
a) If in your machine there are several ‘g++’ named processes (using basically no CPU) and a few ‘cc1plus’ named processes using all the available cpu;
b) If the other node(s) has several ‘cc1plus’ processes;
c) If there is considerable I/O going through the network interface (in my case spikes of up to 2.8MB/s)
d) If icemon shows the jobs moving through the nodes.

I once thought that I’ve configured it all fine and then started ‘make -j30’ with all the jobs running in my machine… needless to say, the machine locked.
🙂

Alright, enough talk, let’s see some numbers… To compile Qt 4.8 from the git repository it takes 34 minutes in macmini, running with ‘make -j6’. By using icecc, and running ‘make -j12’ it compiles in 21 minutes (almost half the time). It is important to remember that there are steps that can not be made parallel (e.g. qmake creating the .pro files, moc running, linking and so on).

For webkit plus Qt5, the numbers are even better: 1h 10m X 19minutes (it is almost 400% faster!). Those 50 minutes *by compilation* can add a lot at end of day.

Finally, it is also possible to combine ccache *with* icecc. To do it, just define your path as: export PATH=/usr/lib/ccache:/usr/lib/icecc/bin/:$PATH. In my desktop, I added the following file in my home:

adenilson@macmini:~/apps/webkit/Webkit$ more ~/.bamf.bashrc
function bamf {
export ICECC_VERSION=/home/adenilson/Desktop/ice_rootstrap.tar.gz
export PATH=/usr/lib/ccache:/usr/lib/icecc/bin/:$PATH
}

Attached you can check how icemon looks like while recompiling Qt 4.8 with 2 different views (starview and gantt view).

Advertisements

November 23, 2012 at 2:58 pm 1 comment

CellarDoor in German plus Debian package

Dear friends

Some quick updates on CellarDoor are due. Since last week, it has debian packaging support in the buildsystem, contributed by Milton Soares.

Simply run ‘make deb’ and you are set: a nice debian package is generated at end.

Next, Adriano Cavalcanti contributed another translation to CellarDoor, German:

Thanks to the fact that German has some bigger than average words, it will require some adjustment in some parts of the UI, but so far so good! As soon I finish some missing features, I’m planning to make a ‘non-hackers’ release.

Currently CellarDoor is available in:

  • English
  • Brazilian Portuguese
  • Italian
  • German

Do you known French or Spanish?
🙂

August 17, 2011 at 1:23 am 13 comments

Presenting CellarDoor

Last January I was talking with my friend Wilson Prata about applications. We have partnered a couple of years ago when creating amora (which by the way needs some care urgently, but this is another matter) and he presented to me a concept for a nice and cool new app: CellarDoor.

What is CellaDoor? Well, it is a wine app. Have you ever started a conversation with someone that also appreciate wines and then failed when trying to remember the name of a good wine you tasted a couple of months ago?

Or what about easily exchanging a wine recommendation? Trying to keep notes in a restaurant napkin is not really much effective.

Trying to address this and other user cases, I started to work in the mockups that he had at time. A great UX designer once taught me the value of close cooperation between programmers X designers to create something that looks good and performs well (and I honestly believe that the best way to create an app is to have at least one designer aboard since the very beginning). This is a different approach than the traditional hack-hack-hack then later ask a designer for some cute icons which is, with some exceptions, the rule in OSS/Freesoftware world.

Unfortunately, I lacked enough free time to progress quickly, but I believe that the app has progressed well enough for a public release. You can checkout the project’s webpage.

So, what are the current features? Here they are:

  • good looking UI with nice usability
  • create a wine card with basic information (year, name, vineyard, etc)
  • filter by wine type (e.g. red, white, other)
  • persistence of data in a sqlite database (which by the way you can edit in the desktop)
  • Supported platforms: Linux, OSX, Windows 7, Symbian 5th and 3 (e.g N8, N5800, etc)
  • snap a photo of the bottle/cork in supported platforms (Linux, Symbian). Persistence of this photo is my todo list (hey, I didn’t say that the app is completely done!).
How it looks like? Check it (running in OSX 10.6):


== What about the technology? ==

The magic to support such varying OSes (Linux, Window, OSX, Symbian), form factors (Mobile, Netbook, Desktop) and resolutions (from 360×640, ranging in 1024×800 and up) is that I used Qt/QML for the UI. At time that I started, Qt Components were not ready yet, so I got to implement my own (probably buggy) widgets like a combobox (based in joint work with my other friend Ricardo Sato) and a calendar widget (coded together with my friend Igor Trindade), this last widget deserves to be pictured bellow:


== What is next? ==

A long backlog indeed waits in the issue tracker of the project. Between those, I believe that next steps should be:

  • Packaging: for symbian I have it done (self signed), Window7 I played with WiX to create an installer but I need help for testing. OSX packaging (.dmg) and rpm/deb help would be greatly appreciated;
  • Translation: it currently supports English and Brazilian Portuguese. I believe it would be cool to support other languages (French, Italian, German);
  • Coding: there are several features that would be awesome to have, if you are interested, let me known.

So, since ATM there aren’t public packages yet, I guess this is a ‘hacker’s only release’. What you need to compile it is Qt 4.7.x and if in Linux, Qt Mobility 1.1.

Concerning the license, it is good and old GPL v2 code with CC non-commercial for the artwork.

August 6, 2011 at 2:01 pm 2 comments

Sliderlayout

It all started with a simple problem: I want to change from one list of things to another, doing a ‘slider’ like transition (like you would do for example in a music player when you choose an artist and go to another list with the tracks).

That sounds pretty simple, right? Just fire a QPropertyAnimation (or if you are using Plasma, a SliderAnimation is even better) and animate the position of 2 widgets.

But what if I want to have those widgets inside of a layout (e.g. QGraphicsLinearLayout)? After all, I want my UI to scale between different window sizes, screen resolutions (and in mobile devices, adjust to screen rotations).

Hum… you could draw your widgets *over* the linear layout and check for resize events of the main widget *while* correctly positioning your set of widgets in the scene. But that sounds a bit too much hack-ish.

I asked 2 friends about this very same issue and one suggested me to create a layout with this behavior. Initially, I was not sure about it, after all, I have never worked before with QGraphicsLayouts and I got almost no time to have something working.

Well, the good news are that it only took me a couple of hours to get this:

[blip.tv ?posts_id=3856759&dest=-1]

Explanation: it is a layout were you can add several widgets and move from one to another by calling next()/previous(). And since it is just a layout, can be inserted in another layouts (in this case, inside of QGraphicsLinearLayout).

Maybe a bit of code would help to explain… say that you have 2 widgets and you want to move next when the current one is clicked:

Item obj1, obj2;

SliderLayout *slider = new SliderLayout;

slider->addWidget(&obj1);

slider->addWidget(&obj2);

connect(&obj1, SIGNAL(clicked()), slider, SLOT(next()));

slider->setCurrentWidgetIndex(0);

and you are done.

The secret here is to keep hidden all the widgets (except by the current one) and in the slider event, animate the position of both widgets (the current and soon-to-be-current widget) while correctly repositioning the widgets in the scene (and setting to show state both widgets of course).

Since I have the movement methods on this layout as slots, I have to derive from both QObject and QGraphicsLayout (and that I’m not completely happy, anyone has suggestions to workaround that?).

I have tested this layout (integrated with *far* more complex widgets) in varied devices (from ARM@430Mhz up to Intel Atom@1.6Ghz))and it behaved pretty well, basically because I’m only animating the position of 2 widgets. It might happen that if you have an overly complex widget, that your hardware is not able to do the animation with good frame rate (but here you can workaround by hiding some of the content of this overly complex widget, and that will be probably the subject of my next post).

There are 2 other remarks about the code:

  • I used Plasma SliderAnimation just because it was easier, but it could be replaced by a QPropertyAnimation
  • Since Plasma Animations expect to receive a QGraphicsWidget, I assumed that you are going to supply a QGraphicsWidget to the SliderLayout. But the real requirement could be a QGraphicsItem, since it is the position that is being animated.

You can get the code here, my personal sandbox for experimenting with plasma animations and other stuff (please, please, we *do* need to migrate away from svn and start using git) and look for src/lib/sliderlayout.cpp and src/example/slidertest.cpp. I wonder when the feature freeze of KDE is over, if this could be part of libplasma.

Maybe this concept could be expanded so there would be a layout class and the animation behavior (fade, rotate, slide, etc) be defined by a delegate. Anyone is tempted to help me to develop this further? 🙂

July 4, 2010 at 9:05 am 1 comment

Lies, big fat lies and benchmarks

There is the old saying, ‘there are lies, big fat lies and there are benchmarks’, but after my previous post about the jsanim, it was time to try to measure how good or bad the javascript based animations are when comparing it against Plasma::Animations (written in C++).

== what to measure? ==

The first task was to have a meaningful number and frames per second (FPS) sounds like a good idea. The only issue is that QGraphicsView will only update parts in the scene that need redrawing (besides it seems that there is not an easy way to access the draw events without subclassing). My solution was to set a counter for the Animation::updateCurrentTime() method that is called while the animation is running, given that the documentation says that “… neither the interval between calls nor the number of calls to this function are defined; though, it will normally be 60 updates per second.”. Maybe I should call it Function calls Per Second (FCPS). 🙂

Another important factor that can impact hugely is this kind of benchmark is both the hardware as also the graphics system used by Qt (and finally if the window manager has composite effect turned on). My test computer is a macbook (version 4.1), Intel Core 2 Duo (T8300@2.4Ghz), video Intel Mobile GM965/GL960), Qt 4.7 from master repository and I run the tests with composite disabled and got this results.

As the test application, I used the code available in playground/plasma/libs/jsanim, where a set of QGraphicsWidgets (each one drawing a pixmap) will run an animation group where 2 animations (zoom and fade) run in parallel followed next by a rotation.

== equal or not? ==

Next task is to extract any meaning from the results and I remembered what an old statistician professor that I had used to joke “Torture the data enough and it will confess”. I used to be good on this subject (back in the day I wrote some Bootstrap simulations and model fitting for biological data as also geostatistics  studies using R), but it is being 5 years since I last have done any statistical analysis, I was worried about making some mistakes while using more advanced methods.

So I went with the lazier approach and decided to use a boxplot, following bellow:

The js based animations seems to have a somewhat bigger variance (see second boxplot) but overall you can’t really tell if there is a difference between them (just in case, the p-values for t-student means tests was for each animation respectively: 0.54, 0.43, 0.86).

So… what about mobile? I did the same tests in a Nokia N900 (using the opengl and raster engines) and got this data, which yielded the following boxplot:

this time without the js variance and again without any sensible difference between js and C++ animations (p-values: 0.42, 0.16, 1).

== wrapping up ==

From the data (and after asking 4 different people to inspect visually how the animations were running in js X C++ apps), I can say that following the path of using javascript to code the logic of animations in Qt seems to be a valid approach with no sensible difference in performance. Kudos to QScriptEngine (and Qt development frameworks) for making it perform really fast.

There are indeed some issues in this study, following:

  • CPU and memory usage: I measure it informally using ‘top’ and they seems to be on the same level, but a more formal benchmarking ideally should measure this variables;
  • samples: I was a bit in a hurry and run each animation only 3 times (ideally a bigger sample like 20 should be used).

April 7, 2010 at 11:14 pm 5 comments

jsanim: current status

From my previous post about jsanim (javascript animations) where I explained the motivations about the idea, I’m writing to report the current status.

I got stuck while trying to create an instance of a javascript class from C++ side in QScriptEngine, but I was helped by khansen (you rock man!).

Being able to create an instance got to address an important issue in the first PoC: run more than 1 animation. Now each animation type will have a javascript class, where its instance will point to the animation target widget and hold the animation parameters. This design has some advantages:

  • uses a single QScriptEngine object (implemented as a singleton);
  • can run multiple animations;
  • multiple animations can access the same widget;
  • is more OOP oriented (the animation loop will call a method on the javascript class);

How fast (or slow…) is it to use QScriptEngine to animate a widget’s properties? What about a video?

[blip.tv ?posts_id=3433718&dest=-1]

Notes: it has 8 QGraphicsWidgets, 24 animation objects (zoom, fade, rotate), running in the Nokia N5800.

What is next is profiling and comparing it against our Plasma::Animation classes doing preferably the same type of work. But since it is still in the very early beginning, it is already looking promising.

March 30, 2010 at 2:00 pm 3 comments

Plasma new animation classes

One of KDE 4 objectives was to create an organic and even more pleasant environment (and it is being achieved with both plasma-desktop and plasma-netbook). One of the features that contributes significantly to achieve a natural look and feel (together with really *great artwork*) are animations.

KDE 4 introduced plasma Animator class with the purpose of applying effects and animations to plasma widgets. With the new Qt 4.6 animation framework (a.k.a. kinetic), plasma effects started to be ported to it, introducing some new animations (e.g. Pulse, Rotation, Stacked rotation, kinetic scrolling) which are already being used in the upcoming KDE SC 4.4.

Kinetic scrolling made its debut in Plasma::ScrollWidget, used internally in uBlog (the twitter/identica plasmoid client) and plasma-netbook (i.e. to scroll through icons in application containment). It has a long story and was rewritten at least 4 times:
– Using a single timer and coordinates for scrolling (with bouncing effect);
– Using percentages for scrolling and properties for position;
– Using coordinates with properties again and implementing the concept of scrolling manager;
– Using QPropertyAnimation to do the animation instead of a timer (and having the bouncing by just changing the easing curve)

The new plasma animation classes has also an interesting story, being submitted through at least 4 big refactorings:
– Initial import based on gSoC project done by Mehmet Ali Akmanalp;
– Animation objects caching;
– Using QAbstractAnimation as base class;
– Reimplementing QAbstractAnimation::updateCurrentTime and non longer using an internal QPropertyAnimation/Group object to actually do the animation;

The good news are that the code is being made more flexible and paradoxically simpler at each review session. What about an example? Say that you want to have that nice pulse effect when a widget name button (i.e.that has QGraphicsWidget as base) is selected you just need to write something like this:


Animation *pulseAnim = Animator::create(Animator::PulseAnimation);
pulseAnim->setWidgetToAnimate(button);
connect(button, SIGNAL(clicked()), pulseAnim, SLOT(start()));

And the same concept is used for the following animations: rotation (2D), fade, grow, zoom, slide, stacked rotation (‘3D’), geometry. Obviously, that depending on the animation type, you got to setup more parameters, like movement direction/reference/distance/axis. Selecting easing curves is also possible, but we are working to have good pre-selected curves that makes sense for each animation class.

Those plasma animations can easily be integrated together with your own animations and directly used in animation groups (i.e. QAnimationGroup) being parallel or linear. Finally, all those classes are being bind with javascript making it dirty easy to have future js plasmoids with nice animations.

So, where to start now? My suggestion is to have a look in kdeexamples where there is a test C++ plasmoid that exercises pretty much all the current available animations.

Next post: video showing the animations. 🙂

December 10, 2009 at 7:59 pm 2 comments


Calendar

October 2017
M T W T F S S
« Apr    
 1
2345678
9101112131415
16171819202122
23242526272829
3031  

Posts by Month

Posts by Category