Tag: hudson

Open Source Love Day December 2009

Our Open Source Love Day for November 2009 brought love for EGit and several hudson plugins. We got slightly frustrated over KDevelop4.

On Tuesday, we had our last regular working day for 2009. We celebrated this circumstance by having our fourth Open Source Love Day (OSLD). The day was successful, you can review our list of today’s achievements below.

We introduced a monthly Open Source Love Day to show our appreciation to the Open Source software ecosystem and to donate back We heavily rely on Open Source software for our projects. We would be honored if you find our contributions useful. Check out our first OSLD blog posting for details on the event itself.

Participate at our OSLD by using the features we’ve built today:

  • Our campfire plugin for hudson was updated to version 1.1. The new version contains the improvements Mark Woods suggested (global configuration and login recovery). Thank you, Mark!
  • The campfire plugin also switched the communication model from webpage scraping to the brand new campfire API. This should improve the stability of the plugin.
  • Some of the EGit (git plugin for eclipse) patches we sent in at the last OSLD needed some rework and polishing. You can review the details in EGit’s code review system gerrit: change 121 and change 122.
  • Our cmake hudson plugin was updated to version 1.1. The new version checks the environment (installed cmake version, etc.) before delegating the call and provides better error messages.
  • We started working on a feature of KDevelop4 that was present in KDevelop3 and is now missing: “Compile file”. The progress was slowed down by some problems. See below for details.
  • Hudson got a new major version of the IRC plugin from Christoph Kutzinksi. The plugin was in a rather desolate state before. We had used a private fork with specific additions to control our infrastructure. The plugin was on our list of OSLD patients, when Christoph merged it with the hudson instant-messaging plugin and introduced a multitude of cool new features. We beta-tested the new version and it was great. The only drawback was the complete alteration in message syntax that broke our infrastructure. So in order to scratch our own itch, we programmed a little API to parse hudson IRC plugin messages of the new 2.X version stream. Our code is published on github, have a look if you are interested and drop us a line if you found it useful.

What were our lessons learnt today?

  • If maven decides to work properly, everything is really cool.
  • Just because you use JGit/EGit on top of Eclipse, all three being platform independent, doesn’t save you from slash vs. backslash issues. EGit’s initial user experience is better on unixoid platforms than on windows systems. The patch #141 helped us beyond the showstopper of unrecognized local repositories.
  • We acquired an additional share of eclipse plugin development knowledge when polishing our EGit features.
  • Working with git and gerrit is challenging on first encounter. We are constantly learning in this area.
  • Bugzilla fails to present open issues in a manner where you can quickly pick an issue of interest. If you really want to use it for your open source project, think of a scraped website that only lists the “low hanging fruits” for newbie developers.
  • KDevelop4 has outdated documentation, the projects kdevplatform and kdevelop were moved inside the repository.
  • If you encounter a rather erratic error stating that “KDE4Workspace not found”, try excluding the debuggers/gdb subproject from your build.
  • Most of us used the waiting delays of one project (“oh, maven is downloading the internet again”) to switch over to a secondary task. So this event trains our multitasking abilities right along.

In summary, this OSLD was a fun way to end a workyear on heavy duty. We will continue to celebrate OSLDs in 2010, as it’s a fun way to peek into foreign projects, learn a lot in short time and contribute to the community.

Speed up your buildbox, Part III: Memory

This is the third part of a series on how to boost your build box without much effort. This episode talks about the effects of faster and more RAM.

© Friedberg - Fotolia.comIn the first and second part of our effort to speed up our buildbox, we replaced the harddisk with a RAM disk and swapped in a bigger CPU. This brought the build time down from 03:30 minutes to 02:00 minutes.

Boosting the memory

When we began the journey, we wanted to undercut the 02:00 minutes threshold. The last component that directly impacts performance of our box was the memory. We started out with 4 GB of DDR2-800 modules. To have a feeling for the effects, we upgraded to 4 GB of DDR2-1066 first and then added another 4 GB, resulting in 8 GB of RAM. We expected the performance gain to be small, but noticeable. The RAM disk, for example, is directly affected by memory speed.

As much, but faster

The first upgrade brought the first surprise: Upgrading from DDR2-800 to DDR2-1066 modules didn’t change anything. It’s not that the mainboard or CPU doesn’t support the faster RAM, it just seems to be fast enough, despite the data bus clock rate. Our build process still took 02:00 minutes, reproducible and without exception.

Filling all the banks

The mainboard can load up to 16 GB of RAM, but our budget just allowed to buy 8 GB of DDR2-1066 RAM. We installed it and ran the same 32 bit Ubuntu Linux as before. The build process took 02:00 minutes, which was expected now.

Changing to 64bit

We changed to boot harddisk, installed a 64 bit Ubuntu Linux and ran the build again. Still 02:00 minutes. The switch to 64 bit wasn’t a big deal with Java, but some of the included native libraries complained about the change. Recompiling them solved the issue.

Finally reaching the target

As a last measure, we increased the maximum memory of the build JVM to the biggest value it would accept. This was -Xmx2600m, a surplus of 600 MB to the original setting. This sped up the build process by five seconds, it took 01:55 minutes now.

Conclusion and perspective

We’ve reached our anticipated target of less than two minutes build time. We exceeded our original budget of 500 EUR, but bought some parts that finally weren’t used in the build box, but elsewhere. The two parts that made the whole difference were the CPU and some more memory to spend it on the RAM disk.

If you want to speed up your single build box, aim for the CPU/RAM combo and try to install a RAM disk to perform all the work on.

This leads me to the perspective of the next part of the series: If you plugged in the most expensive CPU and enormous amounts of RAM to speed up your buildbox, you still aren’t done. You should invest some time to look into distributed builds. Hudson as our continuous integration server provides nearly instant “build slave” support. With this feature, you can set up a whole build farm to further increase your build throughput.

Stay tuned for “Part IV: Beyond the box”

CMake Builder Plugin Reloaded

A few months ago I set out to build my first hudson plugin. It was an interesting, sometimes difficult journey which came to a good end with the CMake Builder Plugin, a build tool which can be used to build cmake projects with hudson. The feature set of this first version was somewhat limited since I applied the scratch-my-own-itch approach – which by the time meant only support for GNU Make under Linux.

As expected, it wasn’t long until feature requests and enhancement suggestions came up in the comments of my corresponding blog post. So in order to make the plugin more widely useable I used our second  Open Source Love Day to add some nice little features.

Update: I used our latest OSLD to make the plugin behave in master/slave setups. Check it out!

Let’s take a walk through the configuration of version 1.0 :

Path to cmake executable

1. As in the first version you have to set the path to the cmake executable if it’s not already in the current PATH.

2. The build configuration starts as in the first version with Source Directory, Build Directory and Install Directory.

CMake Builder Configuration Page

3. The Build Type can now be selected more conveniently by a combo box.

4. If Clean Build is checked, the Build Dir gets deleted on every build

Advanced Configuration Page

5. The advanced configuration part starts with Makefile Generator parameter which can be used to utilize the corresponding cmake feature.

6. The next two parameters Make Command and Install Command can be used if make tools other than GNU Make should be used

7. Parameter Preload Script can be used to point to a suitable cmake pre-load script file. This gets added to the cmake call as parameter of the -C switch.

8. Other CMake Arguments can be used to set arbitrary additional cmake parameters.

The cmake call will then be build like this:

/path/to/cmake  \
   -C </path/to/preload/script/if/given   \
   -G <Makefile Generator>  \
   -DCMAKE_INSTALL_PREFIX=<Install Dir> \
   -DCMAKE_BUILD_TYPE=<Build Type>  \
   <Other CMake Args>  \
   <Source Dir>

After that, the given Make and Install Commands are used to build and install the project.

With all these new configuration elements, the CMake Builder Plugin should now be applicable in nearly every project context. If it is still not useable in your particular setting, please let me know. Needless to say, feedback of any kind is always appreciated.

A Campfire plugin for Hudson

Our last OSLD resulted in a new hudson plugin: a build notifier for Campfire. Right now it is pretty simple and just posts the last build result in a chat room:
Campfire-Notification

You can configure your account data and the room to join in the job config page under Post Build actions:
Hudson-Campfire
But there is more to come:

  • global config
  • SSL support
  • a link back to hudson

or what is your favorite missing feature?

Speed up your buildbox, Part II: Processor

This is the second part of a series on how to boost your build box without much effort. This episode talks about the effects of different processors.

© Friedberg - Fotolia.comIn the first part of our effort to speed up our buildbox, we replaced the spindle harddisk with a Solid State Disk (SSD) and finally, a RAM disk. This brought the build time down from 03:30 minutes to 02:50 minutes.

The Central Performance Unit

The next step on our journey to a faster buildbox was to replace the processor. Our initial processor was an Intel Core2 Duo E6750 with 2.67 GHz. To our pleasure, the processor socket, namely the LGA775 socket, is extremely versatile in supporting different processors. We had no problem in plugging in faster dual or even quad core processors, except upgrading the BIOS.

Taking the 3 GHz mark

The next processor to try out was an Intel Core2 Duo E8500 with 3.17 GHz operating frequency. The L2 cache went up from 4 MB to 6 MB.

The build time went down immediately from 02:50 minutes to 02:20 minutes. That’s nearly 20 percent less build time. And it’s perfectly linear with the CPU speed increase (also nearly 20 percent).

As a result: Investing in CPU clock power seems to pay off. The higher the frequency, the lower the build time.

Doubling the cores

Fortunately, the LGA775 socket supports quad core processors, too. We plugged in a Core2 Quad Q9550 with 2.8 GHz and ran the build again.

The result was astonishing: Despite the lower frequency, the build time dropped from 02:20 minutes to 02:00 minutes. We can’t really explain this one with basic math like the frequency coupling of the dual cores.

If your build is perfectly multithreaded, something javac isn’t, you’ll notice an even bigger speedup.

To sum it up: you can’t have enough GHz or processor cores when running a build.

Reviewing the result

We replaced the harddisk with RAM and upgraded the processor to meet the current performance threshold. This brought us from a starting 03:30 minutes build time to 02:00 minutes now. The CPU is the major player in this game, so upgrade it first.

Outlook on the third part

But what about the RAM? We really wanted to know what happens when we replace the RAM with bigger and faster one. Read more about this experiment in the third part of the series, coming soon.

Our second Open Source Love Day (OSLD)

A retrospective report of our second Open Source Love Day (OSLD). We present the results of our work on hudson and git and the lessons learnt.

opensourceloveday

Today we celebrated our second Open Source Love Day (OSLD). When we say “celebrated”, we actually mean that all of us worked hard and concentrated for hours, just to have a short meeting with candy at the end of the day.

The Open Source Love Day is our way to show our appreciation to the Open Source software development ecosystem. We heavily rely on Open Source products for our customer projects, so it’s just fair to donate back. You can read more about our motivation and specifica in our first OSLD blog posting.

For this day, we adjusted the rules a bit. While Object Calisthenics are very powerful in formulating rather academic software development values in some easy-to-remember rules, they just don’t fit well with existing projects. We still kept the rules in mind, but didn’t follow them strictly. We also learnt our share from last time’s experience of jumping right into the middle of arbitrary projects without a real need to do so. Today, we scratched more of our own itches.

You can participate at our OSLD by using the feature we’ve built today:

  • Hudson gets a brand new plugin. Currently, it’s in alpha status and needs some more nurturing, but is planned to be published within the next few weeks. The proof of concept was successful today. You will read more about it on this blog soon.
  • Another of our hudson plugins, namely the cmake builder plugin, got some feature love, incorporating suggestions from plugin users. We especially thank Ole B. for his feedback. The new features are checked in and will be available with the next plugin version 0.6, scheduled to be published in a few days. You’ll read the details about the new features here.
  • We’ve produced a feature implementation for hudson, adding the ability to use environment variables for the job’s workspace path. This feature touches core hudson functionality, so we just proposed a patch and leave it up to the core hudson team to decide on its inclusion. For more details, head over to the hudson issue tracker, entry #3997.
  • And we didn’t forget about git. As we are multi-IDE users (today’s development took place using NetBeans, IDEA and Eclipse), the EGit eclipse plugin for git will soon have the ability to diff the content of two revisions. An undocumented method argument took too much time to finish the feature today. After email communication with the project owner, the feature works on our machine, but needs some polishing before being committed in the near future.

As you can see, the hudson continuous integration server received a great share of our today’s love. It’s a great tool with a great community that really deserves our contributions.

What were our lessons learnt today?

  • While implementing the variable expansion feature, the author got distracted by a similar concept and followed this red herring. Namely, instead of a hudson.util.VariableResolver, we needed to use the hudson.EnvVars class. The EnvVars are pre-filled with all global variables like JAVA_HOME, while the VariableResolver is not. This could have been avoided by looking at the actual code instead of just type names. Once you think you’ve found your type, you read code the wrong way just to sustain your assumption.
  • To implement advanced plugin features, whether for hudson or eclipse, is a matter of skill with the “monkey see, monkey do” development style. Documentation is mostly non-existent or out-dated.
  • When handling HTML and HTTP in java, some survival tricks are crucial. Stay tuned for a whole blog post on that topic.
  • We still don’t feel comfortable within the JGit source code, as we still lack advanced git feature and terminology knowledge and the project lacks documentation. Our part of the problem will decline over time, as it’s a question of tool/mindset/slang exposure.

To sum it up, this OSLD worked out much better than our first one. We had more fun and yielded better results, mostly because we adjusted our goals to better suit our working style.

What are your experiences with open source development? Drop a comment!

Honey, I shrunk the build box

Meet the world’s smallest hudson server, operating with even less power than your energy saving light bulb.

We are currently posting an ongoing series on how to make your (hudson) build box faster. This article talks about making it smaller.

Making a build box as small as possible isn’t the most familiar requirement today and wasn’t for us. But when i privately bought a fit-PC2, we couldn’t resist trying it out as a hudson server.

The fit-PC2

mini1This is a computer that really fits everywhere. In your car, on the back of your monitor or just, as in our case, on a beer mat. It’s a fully equipped PC with the specification of a standard netbook (Atom 1.6GHz CPU, 1GB RAM, 160GB HDD) and the dimensions of a 5-port Ethernet switch. The most astonishing fact about it is that it uses standard size 2,5″ notebook harddisks. For more information about the computer, look around the CompuLab website, they do not exaggerate.

Operating the fit-PC2

mini2The fit-PC2 is a normal computer in every aspect. We run ours with Ubuntu linux and official Java packages from Sun. As the case is fanless, it accumulates some heat, but never over 60° Celsius (140° Fahrenheit). We measured an average temperature of 45°C on the case surface while building a large project. The Gnome desktop feels snappy, application load delays are sufficiently small and customizing the software outfit is as easy as it can get with Ubuntu.

Setting up hudson

Installing the hudson continuous integration server on a Debian based linux system is a matter of three commands. See Koshuke Kawaguchi’s blog entry on that topic for details. After the automatic installation procedure, hudson already runs on port 8080 of the machine. Setting up the project’s job and initiating the first build were a matter of a few minutes. Hudson reacts swiftly to website clicks.

The world’s smallest hudson server

mini3This is the smallest hudson instance we’ve heard of up to now. It runs in a case measuring 11,5 x 10,0 x 2,6 centimeters. The power consumption is around 8 Watt when building (including the self-usage of the measuring device itself), which would be even lower once we replace the mechanical harddisk with a solid state disk (SSD).

From the performance specifications given above, you should not expect a speed wonder. The fit-PC2 finished the project’s build within 09:50 minutes, which is dangerously near the ten-minutes mark for acceptable continuous feedback. So this box will not go into regular duty, but return home to me (remember, i bought it privately).

Conclusion

The whole purpose of this experiment was to get used to a new era of microcomputers. They are palm-sized and nearly battery operated, but fully equipped with standard components and powerful enough to perform regular tasks. The fit-PC2 is a strong instance of these devices.

Show off your hudson server

Well, to be honest, another purpose of this experiment was to show off our hudson skills, operating with hudson instances from heterogeneous slave farms to this single 300 cubic centimetres box. We would like to hear from your hudson instance. You may add a comment and/or share a link with your story. Maybe the hudson wiki is the ultimate place to gather all the stories.

P.S. This blog entry’s title is an adaption of my childhood’s favorite movie.

Speed up your buildbox, Part I: Introduction & Harddisk

This is the first part of a series on how to boost your build box without much effort. This episode talks about the effects of different harddisks.

W© Friedberg - Fotolia.come actively use Hudson as our continuous integration server software. It has a nice little feature called “build history trend” that shows the duration of all archived builds. One of our major projects started out small and fast with a build duration of 01:20 minutes. One and a half year later, it reached for the 04:00 minute hurdle. It wasn’t a surprise to us, as the build has more than four times the work now and the hardware staid the same.

But a question emerged: How can we speed up our build?

Applying optimization: The basic maths

We did a quick review of our ant build scripts to ensure there’s nothing fundamentally wrong with them and then decided which road to follow first: Optimizing the build scripts or boosting the hardware? There is only one pragmatic answer to us: boost the hardware as long as it stays reasonable in price. Every optimization in the build script would need its time (which isn’t cheap) and possibly increase the script complexity (which is very expensive later on).

Optimizing the hardware

So we went on the journey to make a fast buildbox even faster. We started out with a dual core processor (2.6 GHz), a decent-but-standard harddisk and 4 GB of memory. We replaced every part on its own to see the effect. The journey includes:

Our goal is to cut our build time down by 50 percent, to a little less than 02:00 minutes. We don’t want to spend more than 500 EUR for new hardware. So now, after this introduction:

Part I: Replacing the harddisk

Our buildbox starts with a more or less normal harddisk (0.5 TB), certified for continuous usage. We could have bought just another normal harddisk of a newer generation, but that doesn’t cut it in our experience (we didn’t verify specifically, though).

Calling the carnivores

If you need to upgrade your harddisk, you can buy yourself a VelociRaptor drive and be pretty much assured that you’ll notice the difference. We had pleasant experiences with this kind of fast-spinning drives before, but this time, we wanted to go a step further and try a fast Solid State Disk (SSD). As you only need to relocate the working directories (called workspaces in hudson terminology) of your projects to the new disk, the capacity isn’t important as long as it’s greater than your project sizes. You can just plug your new disk in the buildbox and format it with a high performance file system. As our buildbox runs on Linux, relocating the workspace is just setting a symbolic link. You do not even tell hudson about it. If you happen to run on Windows, check out the “use custom workspace” setting on your job’s configuration page.

An investment of about 200 EUR and 15 minutes of installation later, we had the result: The build average before was 03:30 minutes and now 03:10 minutes. That’s not a big leap forward, as others found out, too. It’s not that the SSD was bad, it performed exceptionally well in the benchmarks, but the harddisk wasn’t the bottleneck. To further proof our assumption, we installed the fastest harddrive you can get: the RAM disk.

Only pretend to use the disk

Linux (like other unixoid systems) has the great feature of an emulated harddisk right in your memory. On Debian/Ubuntu systems, this emulated drive is mounted at /dev/shm and has a capacity of half your total physical memory. It grows dynamically, so you don’t have to worry about its initial size. But you have to check if your workspace fits into it. Our buildbox had 4 GB of RAM and 2 GB were enough to contain the hudson workspace. We configured hudson to build there (you can use symbolic links or the “custom workspace” setting as shown in the picture) and got the result: The build average went down to 02:50 minutes.

custom_workspace

Review on the results

That’s as far as we could speed up our buildbox by just replacing the harddisk. Down from 03:30 minutes to 02:50 minutes, a reduction of 40 seconds or 20 percent. In fact, we even cheated as the buildbox doesn’t use an harddisk anymore for building. With Linux, it’s incredibly easy to utilize a RAM disk as long as you have enough RAM to loan. For Windows systems, there are several software products that can do the same. If you don’t want to loan your RAM, you can look into HyperDrives, but for a price!

So we conclude that the fastest harddisk is an emulated one and even then, its effect on the build time is limited.

Stay tuned for the next episode of our journey to a faster buildbox, when we apply a faster CPU.

CMake Builder Plugin for Hudson

Update: Check out my post introducing the newest version of the plugin.

Today I’m pleased to announce the first version of the cmakebuilder plugin for Hudson. It can be used to build cmake based projects without having to write a shell script (see my previous blog post). Using the scratch-my-own-itch approach I started out implementing only those features that I needed for my cmake projects which are mostly Linux/g++ based so far.

Let’s do a quick walk through the configuration:

1. CMake Path:
If the cmake executable is not in your $PATH variable you can set its path in the global Hudson configuration page.

2. Build Configuration:

To use the cmake builder in your Free-style project, just add “CMake Build” to your build steps. The configuration is pretty straight forward. You just have to set some basic directories and the build type.

cmakebuilder demo config
cmakebuilder demo config

The demo config above results in the following behavior (shell pseudocode):

if $WORKSPACE/build_dir does not exist
   mkdir $WORKSPACE/build_dir
end if

cd $WORKSPACE/build_dir
cmake $WORKSPACE/src -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=$WORKSPACE/install_dir
make
make install

That’s it. Feedback is very much appreciated!!

Originally the plan was to have the plugin downloadable from the hudson plugins site by now but I still have some publishing problems to overcome. So if you are interested, make sure to check out the plugins site again in a few days. I will also post an update here as soon as the plugin can be downloaded.

Update: After fixing some maven settings I was finally able to publish the plugin. Check it out!

We’ve won a prize!

When we switched our continuous integration platform from CruiseControl to Hudson, it was still a younhudsonbutler-149_50pxg project with many white areas on the project roadmap. But it already was powerful enough to handle our settings and delivered real value, so we eagerly wanted to contribute back.

The development process with “release early, release often” policy and community focussed drive fit right into our mindset, so we (in fact, mostly me) spent a few nights figuring out what and how to contribute. The effort materialized in a few private tweaks and a new plugin: the Crap4J hudson plugin.

With the knowledge of Hudson’s internals, we were able to help out various customers to set up their own sophisticated installations (which nearly led to the development of a Perforce plugin when Mike Wille finished his one right on time). This led to various bug reports and feature requests that we filed to Hudson’s issue database.

Soon afterwards, Sun Microsystems announced the GlassFish Awards Program (GAP) as part of the Community Innovation Awards Program. Hudson was part of the participating projects, so I gave it a try and submitted some feature requests and the plugin.

lauriersAnd we won a prize! It’s not the big sort of prize (look at position 50 in the list), but a reward for our filed issues and a honorable mention of the plugin (which truly stands no chance compared to the awesome work of Dr. Hafner, who contributed a complete “get-them-all” collection of useful metrics reporting plugins). At least, we are the only winner from Karlsruhe.

Lately, we blogged about awarding your customers. Well, that’s just what Sun did here. Thanks for that!