Self – Host Almost! Building OpenSolaris on OpenSolaris

I spent the last weekend getting OpenSolaris B99 source drop to build on BeleniX 0.7.1. BeleniX is a compatible OpenSolaris environment containing most of the same packages and the same core operating system base. It does not diverge. I had SUN Studio 11 installed and grabbed the latest stuff from To state in the beginning yes, OpenSolaris is now practically self-hosting with a few simple tweaks – though not out of the box.

After several attempts I did successfully build 99.90% of ON. There are 2 caveats. The 0.09% represents stuff not useful for an OpenSolaris distro – a couple of of WBEM providers for disk and nfs and the DHCP Manager that also depends on WBEM stuff. The Solaris WBEM stuff has not yet been opensourced and will probably be replaced eventually by some other open WBEM implementation. There is a project to bring OpenPegasus onto OpenSolaris. However a DHCP Manager replacement is needed.

Secondly, the ON build requires 2 locale header files that are not separately available in any bundle or gate but are present in (SUNWhea package). They contain some third-party restrictive OSF license and the redistributability is unclear even though they are in the OpenSolaris package repository. I had to snag them from an SXCE DVD I had handy with me since BeleniX does not have these in it’s repo. IMHO it is so crazy to put headers containing a few structs under a restrictive license.

So in a nutshell these are the steps I followed:

  1. Install ON build tools for B99 (I had SUN Studio 11 installed).
  2. Get rid of /usr/gnu from PATH since that generally confuses ON build tools: export PATH=/opt/onbld/bin:/usr/bin:/usr/X11/bin:/usr/sbin:/sbin:/usr/sfw/bin
  3. unalias ls: On BeleniX ls is aliased to ls –color=auto.
  4. On BeleniX you need to have the Java5, devel a few lang packages installed: spkg install java5 devel SUNWuiu8
  5. One needs /usr/ccs/bin/as to be present. This is not a problem since it is a redistributable binary and already present in (SUNWsprot package) and will be provided for BeleniX as well.
  6. So on the OpenSolaris distro you need to execute:  pkg install ss-dev
  7. Copy usr/include/sys/localedef.h and usr/include/sys/lc_core.h from SXCE UGH!
  8. Patch Makefiles to exclude dhcpmgr and wbem providers from build. You can see the patch here.
  9. Modify Makefile.ksh93switch to build ksh93 as bin/ksh
  10. Now you are ready to fire up nightly.

So at least from the SUN’s OpenSolaris distro viewpoint we do have functional self-hosting! From the BeleniX viewpoint it should be fixed soon. The 2 locale headers are the sticking point for non-SUN OpenSolaris distros.

Building binary ON Packages: The OpenSolaris nightly does not build the binary packages. This is due to the fact that usr/src/pkgdefs contain packages that reference closed non-redistributable components. Some of these are entire packages and some containing a few pathnames that reference these closed components. In any case the closed components do not make sense for an OpenSolaris distro, so the packages Makefile and prototype files need to be modified to exclude these.

I used to maintain patches for the Makefile and pkgdefs to do this but that is a maintenance headache requiring manual intervention for almost every build.So on this round I spent some time to come up with a little Python script that can automatically scan pkgdefs in a built ON workspace and patch prototype files and the toplevel Makefile to fix things. You can find the Python script here. So now it is easy to build packages out of an OpenSolaris workspace. I will soon be pushing ON B99 into the BeleniX repo and it will be possible to upgrade the base OS using:  spkg upgrade base

Building XVM source: Along with the on-src tarball I also downloaded the xvm-src piece. and here is how I got that built:

  1. You need a recent version of SUNWmercurial installed.
  2. Untar the xvm-src tarball and follow instructions in sunos.hg/README to setup your $HOME/.hgrc
  3. As before: export PATH=/opt/onbld/bin:/usr/bin:/usr/X11/bin:/usr/sbin:/sbin:/usr/sfw/bin
  4. For BeleniX: unalias ls
  5. One needs /usr/ccs/bin/as
  6. Add in the patches:  ws xen.hg; hg qpush -a
  7. Now the XVM Makfiles appear to contain a bit of hardcoding to point to paths inside SWAN (SUN Wide Area Network). So you need to do the following, otherwise build fails midway:
    1. mkdir /opt/onnv-tools
    2. cd /opt/onnv-tools; ln -s ../SUNWspro; ln -s ../onbld
  8. Now set a couple of environment vars:
    1. export XVM_WS=/path/to/xvm/workspace (where you untarred xvm-src)
    2. export EDITOR=/usr/bin/vi
  9. Finally run sunos.hg/bin/build-all (or sunos.hg/bin/build-all nondebug  for non-debug packages)

Side Note: I put in a small enhancement to lofi(7D) compression which is central to our livecds. Lofi allocates and frees large buffers to hold compressed and decompressed data every time it is called to read blocks from disk. This can cause some heartburn for the VM subsystem. Since lofi use a finite number of max threads to perform I/O it is possible to pre-allocate buffers and re-use them. The buffers can be variable sized so the code keeps note of the high water mark and re-allocates only if a larger buffer is requested. The patch is here and I have submitted an enhancement request #4257.

P.S.: In doing all these I also had to take forced breaks to let my daughter play with GCompris!


3 thoughts on “Self – Host Almost! Building OpenSolaris on OpenSolaris

  1. moinakg Post author

    The memory pressure remains unchanged since at any point in time there are upto a max of 4 active buffers. This change also keeps 4 active buffers. It just reduces the amount of VM activity in having to continuously allocate/free large buffers.

  2. Pingback: OpenSolaris Auto-Builder « The Pseudo Random Bit Bucket

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s