Step 0 to next BeleniX

For the next BeleniX we are planning to pull some binary packages out of OpenIndiana and build some from source ourselves. The primary things we need at this point include the base Gcc 4.5 compiler toolchain and properly working RPM5 packaging. We subsequently need an importer to convert IPS binary packages into RPM5 since we want to pull some stuff from OpenIndiana (like Gnome).

I was playing around with RPM5 on OpenIndiana and now have a working setup, though not properly bootstrap packaged yet. The other major task was to get Gcc 4.5.2 built with all features enabled. It was quite a task given that C++ GMP stuff is involved and that the Gcc 4.3.2 compiler on OpenIndiana is heavily broken and buggy.

Details of how I got to a properly working Gcc 4.5.2 toolchain is here: http://mail.opensolaris.org/pipermail/belenix-discuss/2011-January/001491.html

One interesting new feature in Gcc 4.5 is the support for Link Time Optimizations which begins to match what is being offered by the Interprocedural Optimizations feature in SUN Studio. The Gcc 4.3.2 piece in OI has at least these problems:

1. It provides libssp.so which the compiler looks for libssp_nonshared.a. Libssp is used if compiler options to provide stack protection are enabled.
2. It provides an improperly linked libstdc++ as a result of which C++ exception handling is broken with g++. The library search order in libstdc++ should first be libgcc_s.so followed by libc.so since there exists the same stack Unwind handler function in both with different parameters. The  libgcc_s version is what libstdc++ needs to use.
3. It is not able to compile the Parma Polyhedra Library required by Gcc to support loop optimizations via advanced Polyhedral methods. Some C++ template handling seems to be broken.

In any case now that I have the Gcc 4.5.2 toolchain working properly I would be re-building RPM5 with that and creating a bootstrap framework next.

 

OpenGrok Server

Given that not many realize the powerful features and polished interface that OpenGrok provides, I decided to put up an OpenGrok server to cross-reference some common major FOSS software stacks on a development machine used by BeleniX folks: http://belenix.v12.su/source/. It contains stuff like the Linux kernel trunk, Python, Ruby, OpenJDK, Glibc among others.

I also have put up patched kernels corresponding to Fedora 13 and 14 releases. It was in interesting exercise to automate getting patched Fedora kernels. Since I wanted as much of history info as possible I did not use tarballs. Rather I checked out the base kernel patch release from Git and applied all the patches in order as listed in kernel.spec and then I did a git pull to merge the fedora kernel git bundle into a subdirectory in the main kernel tree. You can grab the script from here: http://www.belenix.org/binfiles/get-fedora-kernel.

I will add some more stuff shortly. In addition it will take me a few more days to setup an auto-sync mechanism to keep the source trees synced to upstream every couple of days. For now the tress are being manually updated. Hopefully this will prove to be an useful resource going forward.

 

Network Installer for OpenIndiana

I had to upgrade a build server running BeleniX 0.8 recently to OpenIndiana. This was needed to move to a more updated environment for building the next BeleniX release. Obviously BeleniX 0.8 being a custom-built distro based on SVR4 packaging had no upgrade path to an OpenIndiana release based on IPS.

What I actually needed was to have OpenIndiana installed into a new ZFS boot environment and boot from it which would allow me to go back to BeleniX 0.8 if I needed it. I of course had the same capability in BeleniX via the Network Installer that I had written earlier. Now I needed the same for OpenIndiana. So I spent a week modifying and experimenting and I now have a network installer for OpenIndiana. You can grab it from http://www.belenix.org/binfiles/install_openindiana.

Obviously I will have to update the BeleniX version once the next release is out since we will be moving to an RPM5 based packaging. This network install technique can be adapted to any distro and will allow a multi-boot setup based on ZFS boot environments. Apart from this there are other possibilities that I can think of. At present the network install script uses the package collection from the slim_install package group. A new package group for a non-GUI base environment can be leveraged via this script. This can also play a part in a minimal CD based environment to quickly install predefined setup, something not unlike the Automated Installer but without all that complexity. If gPXE can be properly made to handle booting an OpenSolaris kernel then it will be possible to deliver such install environments over HTTP for eg.

 

Govt’s Green Agenda of the decade!

I have a lot of respect for our outspoken and generally level headed environment minister Jairam Ramesh. However as ministers go they all have to say something weird every once in a while. I am referring to this news item from the Times Of India: http://timesofindia.indiatimes.com/business/india-business/Discourage-sale-of-BMWs-Audis-says-Jairam/articleshow/6913269.cms

Some of the statements like fuel efficiency norms are quite valid but, with all due respect, other statements only make a very partial sense at best. There are a zillion things to do to reduce carbon and other effluent emissions and pollution in general in India. Trying to stomp down on modern day fuel-efficient, emission norms compliant vehicles is basically the least of those. I think the following have more priority:

• Why not stomp down first on carbon and pollutant belching and diesel guzzling years old trucks. One of these would be pumping out pollutants equivalent to 50 AUDIs put together every day.
• Why not stomp down on the minivans, matadors, upgraded autorickshaws, country jeeps, tempo traveller vans, mini trucks, country motor fitted carriages and sundry other vehicles of every description plying recklessly in Indian cities. Some of these are even fitted with pump-derived motors and they belch clouds of black smoke to their hearts content.
• Why not maintain older Govt buses properly so that they do not belch pollutants including monoxide. Some authorities like in Bangalore are using more and more of latest Volvo buses but the authorities need to introduce CNG on war footing.
• Why not force all autorickshaws to use CNG or at least an LPG conversion kit instead of using Kerosene mixed with motor oil as fuel and belching dangerous pollutants.
• Why not strictly enforce fines for all vehicles belching black smoke on the roads instead of simply catching 2-wheelers for document checking or cars for signal violation.
• Improve road quality and traffic management – better roads and better traffic management lead to smoother vehicle movement and reduced overall fuel consumption and emission.
• Why not regulate industrial pollution and emissions. Just as an example several of Bangalore’s large lakes are have been converted into toxic cesspools by scores of unregulated garment factories spewing tonnes of industrial effluent into them. These effluents include persistent chemicals that form long-lasting foam when exposed to light and air. These foams coat the inlet and outlets to these lakes and refuse to go away.
• Why not regulate use of plastic bags especially the thin toxic locally-made ones.
• Why not regulate use of country-made 4x4s with no emission norms at all..

There are many others but I listed the ones I thought were most important and in context.

Respected minister sir I’d consider all these as criminal destruction of our environment rather than use of high-end Audis and BMWs. How many of these high-end SUVs get sold in a year ? Does it really make sense for 50 Audis not to get sold in a calendar year when other weirdo vehicles are pumping in hundreds of tonnes of emissions daily ?

 

Howdy OpenIndiana

I have recently started playing with OpenIndiana as part of renewed effort in the BeleniX space to realign with the latest developments. We intend to collaborate and participate in the OpenIndiana community. We are still firming up the plans and approaches.

In the meantime I have been playing with OpenIndiana a bit and find it quite appealing once the few initial hiccups are worked around. They are minor in any case. It is fast and smooth especially the speed with which Firefox loads. This is how the OpenSolaris distro project should have been setup I feel, driven and “owned” by the community from which SUN could have derived value. However communities were formed and then messed up. For example http://lwn.net/Articles/370157/, and BeleniX efforts which were broadly leveraged to build a flagship product and then the original source summarily ignored.

However that is all past now the organization having succumbed to the many problems that riddled it. Some things stand out like diversity issues and lack of business innovation.  There are interesting developments happening now in the fledgling community around Illumos and OpenIndiana. I am excited by all that and motivated to participate and contribute.

OpenIndiana in all this is really looking good and I highly recommend it for folks to try out. The technical credit no doubt goes to the erstwhile SUN engineers around distribution, OS, install, desktop and related groups who put in countless hours of effort. Credit also goes to folks like Alaisdair, Guido for getting the community to rally round all this, organizing and actually delivering something excellent which was earlier missing in action. This is just the start though and a lot of concerted effort is needed by the community to keep this going and developing. Lets hope for the best.

 

OGB Candidacy Details

On the lines of the excellent bio by Peter Tribble I am posting my own bio here as required by the OGB candidacy rules.

DECLARATION OF INTERESTS

1. I am Platforms Engineer cum OS and apps Developer presently employed with Goldman Sachs. Goldman uses SUN Solaris and Redhat Linux among other platforms in the firm. However my engagement with the OpenSolaris community is a hobby activity that I do outside of my work with my employer’s consent. As such all my views and actions are entirely my own done using personal resources and time and not related in any way to my employment.
2. I have been a user and developer on Solaris and Linux platforms for the last 12 years (of a total industry exposure of 13.5 yrs) and have been participating in the OpenSolaris community from it’s early days. I have previously been employed by SUN in the Solaris Sustaining engineering group working on various aspects of the OS from userland to the kernel.
3. I am also the creator of the BeleniX distribution of OpenSolaris: http://www.belenix.org/. It was the second distribution of OpenSolaris that came out after SchilliX borrowing some concepts from SchilliX. It was the first non-SUN OpenSolaris distro to bring a full-fledged GUI desktop based completely on the X.org OSS stack and eventually matured into a stand-alone desktop distro. It brought in several innovations to OpenSolaris and formed the foundation for the OpenSolaris distro from SUN.
4. I am a core contributor in a few OpenSolaris communities like X-Windows, Distribution etc. I contribute to the Fully Open X project off and on and have recently started another project called libtaskq (http://sourceforge.net/projects/libtaskq/) based on the TaskQ kernel framework from OpenSolaris.
5. I co-lead one of the oldest and very active OpenSolaris user groups, the Bangalore OpenSolaris User Group with another OpenSolaris community member Sriram Narayanan.

BRIEF BIO
I live and work in Bangalore, India’s Silicon capital. However I was born in the eastern city of Calcutta which was once the capital of the British rule in India. I did my studies in Calcutta at Asutosh College under the auspices of the University of Calcutta. However I do not come from an Engineering background. My majors in Graduation were Geography, Geology and Economics while I studied Economics, Statistics and Maths in high school. I had a deep interest in Biology and Geomorphology till high-school and actually wanted to do Biotechnology as a career!

However I developed an interest in Computing as a hobby during the March of 1990 (thanks to my mom) and my first exposure was on the BBC Model B microcomputer which I hacked to death at my Mom’s office – Birla Industrial and Technological Museum. After that I completed all the typical topics of a Computer Enginering course as a hobby while studying Geography and moved from the BBC to a PC-AT and all the subsequent Intel processor models.

After hacking around with Borland Pascal, C/C++, Win32 etc. my first introduction to *nix was on Slackware Linux 0.1. By that time I have completed a PG diploma course on Software Engineering and my first job had me working first on FoxPro and then on Oracle on WinNT.

My second job provided me a big break when I joined HCL Technologies in the southern tropical city of Chennai where I started working at the dedicated Cisco offshore development center. That was the time when I came into touch with Solaris 2.5.1 logging onto large engineering servers via big-screen TektroniX X-Terminals. That experience at HCL – Cisco provided me with a wealth of resources and expertise. I later started having my first SPARC desktop and SUN Ultra 5. I worked across various Cisco groups including Test Automation group, Network simulators, Network Management group with my work touching a vast array of computing technologies starting from router chips and OS platforms and continuing till Java and webservices frameworks. I played with the guts of routers costing hundreds and thousands of dollars apart from a variety of SUN Servers.

After my 5.5 yr stint at HCL – Cisco I decided to accept an offer at SUN Microsystem’s Solaris Sustaining Engineering group and worked there for 4.5 yrs till the middle of 2008 when I jumped ship to Goldman Sachs in their Platforms Engineering group. In SUN I worked on various pieces including, commands, libraries, systems management and a few kernel components as part of my OpenSolaris dabblings.

I have been a voting member of the OpenSolaris community from some time.

 

Musing in the SUN – Set

Now with the EU approval coming the curtains have started coming down on the acquisition soap opera that has been going one for the past year or so. There are still a couple of regulatory approvals pending from Russia and China but the major hurdles have been crossed so that the assimilation process by Oracle can begin in earnest. This is good news for all SUN customers and users as the uncertainities end and roadmaps start to be delivered.

With this a decades long fantastic story of innovation and technical brilliance comes to an end, SUN as we have known it begins to set. Hopefully Oracle will keep the core essence alive and we will continue to see great developments happen on the most popular SUN platforms, Solaris, Java, Systems etc. Hopefully we will continue to see the opensource efforts continue and OpenSolaris community, source bases continuing to flourish.

It is both a time of relief and sadness for me seeing the acquisition process finally moving forward. I am proud to have been among the privileged denizens of the hallowed portals of this great organization. I worked for SUN for 4.5 yrs leaving it in the middle of 2008. Those years were probably the most exciting and fulfilling years of my career that I have had till date and will shine in my memories till the end of my days.

Leaving SUN has been one of the most excruciating decisions I was forced to take till date. It is akin to what an old Oak or Pipul tree might feel if it is transplanted. I was literally torn from my roots. I  still cherish the teams and the people I had worked with. The experience at the SUN workplace was so rewarding for me in a host of ways that I essentially regarded it as my second home. I will probably never get to experience such a fascinating workplace again: amazing people, exciting technology and projects, culture of innovation, work life flexibility, efficacy of people management, lack of hierarchy in perception and behavior of management, boundless opportunity to learn, freedom to define and create your own projects and work boundaries. These were some of the unparalleled qualities that I am yet to see anywhere else. My frustrating experience around lack of recognition of my BeleniX work (that provided the foundation for the OpenSolaris distro), towards the fag end of my SUN career was but a tiny fly in an otherwise spotless ointment.

Many will of course will have their own views and ideas on why SUN failed in spite of having some of the best technology talent in the industry. Since I have experienced the place first hand, let me put down a few pet views of my own:

• SUN has been and still continues to be the crucible of brilliant technology innovations but it lacked in business acumen and leadership. In this context I came across this thought-provoking article: http://blogs.harvardbusiness.org/anthony/2009/03/what_makes_a_company_the_world.html. I am hopeful of the future of the core SUN technologies under Oracle’s business acumen that SUN direly lacked.
• Pointless acquisitions.
• The OpenSolaris distro and the way some of the community matters have been handled. While the distro has been great for visibility and adoption I personally fail to see the justification of putting crucial time, money and effort into developing an opensource distro without a proper business model. Why would one put their business and top engineering resources into developing something without a revenue model ? Which enterprise customer will be willing to deploy a software from a development trunk into their data centers where they host business critical applications ? I am hopeful of Oracle here again. SUN customers would really like to lay their hands on Enterprise Solaris 11. Of course an opensource distro is needed so SUN should have followed the Fedora -> RHEL model. Empower the community and let it deal with distro things and focus your business resources in doing, ahem, business. There were efforts out of the community like BeleniX, SchilliX and others that could have fit the bill. Obviously BeleniX being my creation I am biased towards it. However BeleniX indeed provided the full foundation of OpenSolaris distro so it could have evolved, at that time, into the community distro variant like Fedora. However it was ignored after the benefits were extracted.

There are many obvious others I have skipped here. Anyway I am looking forward into the future with an expectation that Oracle can recover the market that Solaris and OpenSolaris lost in recent times. Solaris today is probably the most mature and technically capable OS out there in the wild and it deserves to be more widely adopted than it is today. While people only look at the blockbuster ZFS, Dtrace and Zones stuff, it is really full of compelling technologies that together make it a compelling platform: MDB/KMDB, Proctools, Truss, Apptrace, CTF, Resource Management, CIFS, Seamless Real Time extensions, Crossbow networking, I/O Multipathing, SMF and so on. Joerg Moellenkamp has an excellent brain dump of all these “other” features in his open book “Less Known Solaris Features“.

 

Building Packages on BeleniX #1

Several people have been asking me to detail the steps I use to build packages for BeleniX, so here is an overview of what needs to be done and what are the gotchas to be kept in mind.

BeleniX uses the Pkgbuild utility to build packages. Pkgbuild is similar to Rpmbuild and uses Spec files to store the build recipes. However instead of generating RPMs it generates SVR4 packages. Since we are actively considering a move to RPM from the venerable but somewhat dated SVR4, having spec files that need a little massaging is an advantage. Though this is still being discussed a bit. If you are not familiar with Spec files I’d recommend you to go through some of the excellent resources and howtos available on the net regarding writing Spec files and generating RPM packages. In addition Pkgbuild has some OpenSolaris – specific extensions that are documented in the Pkgbuild manual page.

However you should Not use the standard Pkgbuild utility from the original website. We have a slightly modified older version of Pkgbuild that supports the “Conflicts” directive among a few other tweaks. This is packaged via the BELpkgbuild package available in the BeleniX package repo.

If you have installed BeleniX 0.8 Beta1 using the install_belenix Network Installer mechanism, you will find that the installation lacks various headers, compilers and other files needed for development. To ease the task of getting a baseline development environment the package repo provides a group package called “devel” that pulls in all the necessary stuff. Consider this as the Debian “build-essential” thingy.

In BeleniX we primarily use the Gcc4 compiler since in the spirit of opensource we much prefer an opensource toolchain even though SUN Studio can provide some benefits over Gcc in terms of performance. In addition it is much easier to port and build third-party software using Gcc as compared to Studio since FOSS developers tend to use Gcc themselves. Since we are a resource-constrained small team we would prefer not to spend all our time chasing down compiler related build issues. The “devel” group package pulls in Gcc4 as well. However in addition to Gcc4 you will need Studio for certain things like building OpenSolaris from source. In that case I’d recommend you to download and install the latest tarball version of the SUN Studio compiler by visiting this page.

Next you will have to checkout the BeleniX spec files from the SVN repo and setup the initial Pkgbuild environment.

So preparatory steps are:

1. Install development packages:  pfexec spkg install devel
2. Install SUN Studio(optional)
3. Checkout spec files from BeleniX SVN: svn co https://belenix.svn.sourceforge.net/svnroot/belenix/trunk/spec_files belenix_specs
4. The above command may give an error due to an issue older locale packages on the beta1 release try setting the C locale explicitly: export LANG=C
5. Now create some directories that Pkgbuild expects. By default Pkgbuild expects a directory structure within the user’s home directory. The following directories need to be created: ~/packages/SPECS, ~/packages/SOURCES, ~/packages/BUILD, ~/packages/PKGS
6. Now copy the needed files into these directories:
   1. cp belenix_specs/base-specs/* ~/packages/SPECS
   2. cp belenix_specs/include/* ~/packages/SPECS
   3. cp belenix_specs/patches/* ~/packages/SOURCES
   4. cp belenix_specs/ext-sources/* ~/packages/SOURCES

Now you are ready to play around further. Typically Pkgbuild provides two utilities called pkgbuild (aka rpmbuild) and pkgtool a higher-level wrapper around pkgbuild that can automatically download sources files and execute all the pkgbuild steps to generate a package and also install it. You can read more about pkgtool and pkgbuild commands by executing them with the ‘–help’ option or read the online manual. i am not going to repeat the details here.

In order to correctly use pkgbuild however certain environment variables and PATH must be set. At the minimum CC and CXX variables must be set to point to the C and C++ compilers. To simplify things a wrapper script is provided that sets up things properly for SUN Studio or Gcc4. It can be found at belenix_spec/run-build assuming belenix_specs is where you have checked out the spec files. So to build a spec file using Gcc the incantation will look like this:  cd belenix_specs; sh run-build SFEfoo.spec gcc

The above invocation will automatically download the tarballs, apply patches, build, package and install the package. The SVR4 packages are kept under ~/packages/PKGS. During the build process the build output is sent to a log file having the package name. So for SFEfoo.spec the log file will be /tmp/SFEfoo.log. However pkgbuild first sends the logs to a temporary file and then appends those to SFEfoo.log. In order to monitor a running build the following simple script can be used:

#!/bin/sh
PID=`ps -ef | grep pkgb | grep " /bin/bash " | awk '{ print $2 }'`
LF=`pfiles ${PID} | grep "/tmp/pkgbuild.out" | head -1`
echo "********************"
tail -f ${LF}

This completes a brief intro into the initial preparatory steps. The next step is to actually look at a spec file and prepare a new one. There are various gotchas and things to keep in mind when doing that. I will briefly go through the stuff that one should do when preparing a new spec file on BeleniX in the next post. In the meantime one can look at existing spec files to check how things are done. Two suggested sample spec files to look at will be SFEpth.spec and SFEmrxvt.spec. Pth is the GNU portable threads library. It shows how to write a single spec file to build both 32Bit and 64Bit versions of a library. Mrxvt is a multi-tabbed terminal emulator. To look at typical cmake invocation that I have been using one can look at the KDE4 spec files in belenix_specs/kde4.

 

The loyal Dog on my PS3

I had got a PS3 a few months back with a multitude of objectives. Blu-Ray player for my planned Home Theater setup, general media center, watching Youtube on the big screen, Gaming of course and to hack on the Cell. For the last one piece fortunately I have the 80GB Fat/Phat model.

I had been holding out on the actual Linux installation and hacking since we experience power cuts and a power loss during a system update or Linux installation can leave me with an expensive brick that my wife will probably be prone to break on my head!

So I had to invest on a UPS. I already had an APC Back-UPS RS 1000 with additional battery backup for my PC. I decided to get a beefier Smart-UPS 1000UX with 100AH external batteries for my PC and hook up the PS3 to the Back-UPS. However the LCD TV is on line power and when power goes it goes dark naturally and I can’t switch off the PS3 without having a working display. I can hook the TV to the UPS but I do not want the UPS to be always powering the TV. So I whacked together a double-pole 2-way switch. I put 2 modular single-pole 2-way switches side by side and stuck 2 pvc pieces with superglue on either end. The common lines from these switches go to the TV. One set of poles connect to UPS power and one set to mains. Ordinarily the switch connects to mains. When power goes while PS3 is in use I can flick it to put the TV on UPS battery power. Admirably the Back-UPS can take a sudden load surge while on battery without tripping. The UPS has already saved me once.

Then I went about the process of formatting the PS3 for Linux and installing the kboot loader. Initially I decided to go with Ubuntu. Version 9.04 did not work with the latest 80GB console models so I tried 9.10. The GUI came up with that and it hanged trying to sync with NTP servers. After googling I decided to pull out the Ethernet cable from the PS3 while installing (I have a concealed CAT-5 cable running from PS3 in living room to a switch in another room). The installer proceeded beyond NTP server sync and hanged again. I decided to give up on Ubuntu and go with Yellow Dog Linux 6.2. I had avoided YDL initially since it is a large 3.4GB download. The experience with YDL was extremely smooth. It installed flawlessly and works quite fast. I have a happy puppy yapping on:

YDL 6.2 installation in progress

Post install configuration

Firefox and terminal running in E17

 

Impressive Fedora

I use Linux as well apart from my obsession with OpenSolaris. I have used several distros in the past and came to like Mandriva for general use. I was also once the biggest critic of Fedora. Having had bad experiences with FC3 and FC4 I cursed it and simply ignored it till recently when I started stealing spec file recipes and patches from FC 11,12 CVS repo for building packages on BeleniX

During the course of that usage I now stand to take back my earlier criticisms of Fedora. In fact I am thoroughly impressed with the quality of the work they are doing. The quality of the spec files and patches speak for themselves. Many of the conventions they follow align with how things are laid out on OpenSolaris as well. I am now going around and recommending Fedora 12 to anyone who is using Linux. The only issue that one will see is from a desktop use perspective. Out of the box Fedora has very few customizations and tweaks, so it takes a while of manual work to tune it to your liking.