Building GLib With Sun Studio Tools - A Case Study

By Nikolay Molchanov, Sun Microsystems, May 10, 2006

GLib is a general-purpose utility library, which provides many useful data types, macros, type conversions, string utilities, file utilities, a main loop abstraction, and so on. It works on many UNIX-like platforms, Windows, OS/2 and BeOS. GLib is released under the GNU Library General Public License (GNU LGPL). On UNIX, GLib uses the standard GNU build system, using autoconf for package configuration and resolving portability issues, automake for building makefiles that comply with the GNU Coding Standards, and libtool for building shared libraries on multiple platforms.

We'll use Sun Studio 11 compilers and tools to build the GLib library on Solaris. Sun Studio 11 is a complete development toolset for software development on Solaris (SPARC, x86/x64) and Linux (x86/x64). It contains C, C++, Fortran compilers, assembler, debugger, performance analyzer, distributed and parallel make, full set of documentation, and an Integration Development Environment (IDE). It is a supported Sun product and it is FREE. Anybody can download it from www.sun.com and use it without charge. Sun Studio 11 is used to build OpenSolaris (www.opensolaris.org/os/), and it is the development toolset of choice for development on Solaris. So we decided to try to build Open Source applications and libraries with Sun Studio 11 to demonstrate the following:

Is Sun Studio suitable for this purpose?
What kind of build problems will we uncover?
How does the Sun Studio 11 "dmake" (distributed and parallel make) tool compare with GNU "make".

Getting Started

We used a Sun Fire V40z Server, configured with 4 single-core 2.4 GHz AMD Opteron processors, 8 Gb memory, and 200 Gb of local storage, running Solaris 10. We downloaded the GLib library (glib-2.8.6 from ftp.gtk.org/pub/gtk/v2.8/), and extracted the sources using "gtar" from the /usr/sfw/bin directory (file /usr/sfw/bin/gtar is GNU tar; it is shipped with Solaris). Then we installed Sun Studio 11 in the default location.

After that we were ready to start the build.

The normal sequence for compiling and installing the GLib library is:


          ./configure
          make
          make install

The default "make" on Solaris is /usr/ccs/bin/make. The default Solaris "make" tool supports POSIX "make" syntax, and SUN and SVR4 extensions. It does not however support GNU extensions. This is not an issue for GLib, because it can be configured for different "make" tools. We created a "bin" directory, and set PATH so, that our "bin" is the first in the PATH:

export PATH=`pwd`/bin:/usr/bin:/usr/ccs/bin

This "bin" directory contains symbolic links, that allow us to try the following "make" tools:

dmake -> /opt/SUNWspro/bin/dmake gmake -> /usr/sfw/bin/gmake make -> gmake solaris_make -> /usr/ccs/bin/make

Build with GNU make.

For the first build we made the symlink "make -> gmake". Unfortunately the build failed at the step "configure", because it did not find a "perl5" binary. To correct the problem we created one more symlink in our "bin" directory:

perl5 -> /bin/perl

The build was successful (time make). It took 42 seconds to compile all files and to create the library:

real 41.6
user 14.6
sys 3.3

Build with Solaris make.

For the second build we used the standard Solaris "make" utility. We changed the symlink, extracted sources and started ./configure. Now "configure" created makefiles with Solaris "make" syntax. The build was successful, and it took 44 seconds to compile all files and to create the library:

real 44.1
user 14.9
sys 3.7

Build with Sun Studio 11 dmake.

For the third build we used the Sun Studio 11 "dmake" utility in parallel mode. We changed the link ("make -> dmake"), and set the environment variables:

export DMAKE_MODE=parallel export DMAKE_MAX_JOBS=16

These settings allow "dmake" to run 16 jobs in parallel. Again, we extracted sources and started "./configure" and "time make". The build was successful, and it took 18 seconds to compile all files and to create the library:

real 18.0
user 15.3
sys 4.9

Room For Improvement.

Note that the "real" time is smaller than the "user+sys" time, which means that several jobs were running in parallel. In the ideal case, if the makefiles allow us to run all jobs in parallel, the "real" time should be 4 times faster than the "user+sys" time on a 4 CPU system. So, we looked at the generated makefiles, and found out that there are several constructions, that "block" the parallelization. Here is one of them:

# This directory's subdirectories are mostly independent; you can cd
# into them and run `make' without going through this Makefile.
# To change the values of `make' variables: instead of editing Makefiles,
# (1) if the variable is set in `config.status', edit `config.status'
#     (which will cause the Makefiles to be regenerated when you run `make');
# (2) otherwise, pass the desired values on the `make' command line.

$(RECURSIVE_TARGETS):
    @set fnord $$MAKEFLAGS; amf=$$2; \
    dot_seen=no; \
    target=`echo $@ | sed s/-recursive//`; \
    list='$(SUBDIRS)'; for subdir in $$list; do \
      echo "Making $$target in $$subdir"; \
      if test "$$subdir" = "."; then \
        dot_seen=yes; \
        local_target="$$target-am"; \
      else \
        local_target="$$target"; \
      fi; \
      (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \
       || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \
    done; \
    if test "$$dot_seen" = "no"; then \
      $(MAKE) $(AM_MAKEFLAGS) "$$target-am" || exit 1; \
    fi; test -z "$$fail"

So what is wrong with using "for ... do ... done" loop in makefiles? It breaks parallelization.
The rule "for ... do ... done" is a shell script. For any "make" it is just one shell command that should be executed, so there is no way to build in parallel any of the targets that are combined in $(SUBDIRS).

There is a standard way to rewrite such loops using the "make" language. Here is an example to illustrate the idea.

This is a simplified rule to build $(RECURSIVE_TARGETS), written using shell "for" loop:

$(RECURSIVE_TARGETS):
    @set fnord $$MAKEFLAGS; amf=$$2; \
    target=`echo $@ | sed s/-recursive//`; \
    list='$(SUBDIRS)'; for subdir in $$list; do \
      echo "Making $$target in $$subdir"; \
      (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$target) \
       || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \
    done;

This is the same rule to build $(RECURSIVE_TARGETS) written using "make" language:

$(RECURSIVE_TARGETS):
    target=`echo $@ | sed s/-recursive//`; \
    $(MAKE) $(AM_MAKEFLAGS) TARGET=$$target;
$(SUBDIRS): FORCE
    echo "Making $(TARGET) in $@";
    (cd $@ && $(MAKE) $(AM_MAKEFLAGS) $(TARGET) );
FORCE:

Looks simple? Yes, we think so. But is it enough to build in parallel all targets that are combined in $(SUBDIRS)? Almost. The only thing that is missed is the following line:

.PARALLEL: $(SUBDIRS)

This allows us to build in parallel all targets from $(SUBDIRS). Can they be built in parallel? According to the comments above, this directory's subdirectories are mostly independent, so it seems that the answer is yes. But when we tried to build them all in parallel, we found out that there are hidden dependencies. This is not a problem, the dependencies should be specified in makefile to help the "make" utility understand which targets should be built first:

# Dependencies tests: gobject gobject gmodule gthread: glib