I hammered out the BeleniX 0.7.1 release recently after quite a bit of work. One of the unfortunate things is the inexorable growth of the Ramdisk size which is used as the root filesystem when booting off CD. Obviously the OpenSolaris kernel has been getting more and more new features and drivers which means more and more kernel modules. In addition the Nvidia driver weighs in at a hefty 7.5MB for a single kernel module!
For all these reasons the ramdisk in 0.7.1 is around 84MB in size when uncompressed. This inexorable increase in size means that it will be more of a problem going forward when trying to run the livecd on low-memory machines. In addition it is also a problem for distros like MilaX which try to create an image that is as small as possible.
The ramdisk in 0.7.1 would have been even larger had it not been for a small tweak . There are many modules under /kernel that are not necessary for booting. These can be easily placed under /usr/kernel which will reside in the compressed file freeing up space in the ramdisk. The BeleniX Constructor now relocates modules given in a list from /kernel to /usr/kernel. However these modules are moved back to their original locations during harddisk installation to maintain consistency in the installed image. This list is not complete and more modules can be added.
In addition it will be nice to be able to compress files in the ramdisk akin to SquashFS. There is an ongoing SquashFS porting project on opensolaris.org. However it will also mean making the OpenSolaris kernel bootable off SquashFS. Recently I came to know of a little-known piece in OpenSolaris called Dcfs. A little OpenGrokking leads me to the following source code: onnv-gate/usr/src/uts/common/fs/dcfs/. It is just a single file and comments at the top explain all. This is a shim pseudo filesystem designed to work in conjunction with UFS. This adds an on-the-fly decompression layer above UFS. This is read-only compression and a command line utility called fiocompress is used to compress the files initially. This feature was added last Dec as part of the SPARC Newboot project that makes booting on SPARC similar to that on x86 using a boot_archive or ramdisk. That project employed this Dcfs feature to reduce size requirements of the ramdisk (very much needed since RISC binaries are bigger). So Dcfs can be used for BeleniX as well.
Now the interesting part came up when I spent half an hour reading through the code. The compression algorithm is identical to what is being done in Lofi Compression! The same header, index and segmented compression structure. Portions of the fiocompress code bear resemblance to the compression code in lofiadm. In addition Lofi Compression was introduced more than 2 years back in Feb 2006. This leaves no doubt as to where the algorithm came from even though the exact implementation is a little different. I am very happy to see my innovation making it’s way into other parts of OpenSolaris and adding value.
However this knowledge also caused me some grief. All this was done silently giving no attribution to the original source. Even the comments in the Dcfs file does not mention where this algorithm comes from! This is unfortunate. I’d have expected the courtesy to give recognition/attribution where due.
July 23, 2008 at 8:02 pm
Very interesting. I tried to compress image with solaris squashfs but did not receive significant superiority compared with lofi. But looking at SliTaZ think that progress is possible
July 29, 2008 at 1:30 am
The fact that the lofi compression was not referenced in dcfs is regrettable… as it’s father, I know you must feel very proud to see so many of your contributions being used in opensolaris (even if a lot of them don’t give proper credit..) and I want to congratulate you on your excellent work on Belenix.
I hope you and the rest of the Belenix group can continue to give more of such excellent “children” that make solaris a much better OS than what it already is.
July 29, 2008 at 1:43 pm
Thanks!
Yes of course it is matter of pride. However the fact is that zero credit has been provided in every place. My 2+ years of work on BeleniX and direct contributions to Project Indiana provided the foundation on which OpenSolaris the distro stands today. Yet not a single recognition of that fact has been made anywhere in any reference to OpenSolaris.
It would not have been possible to get the first beta preview release out last year with just 4 months of effort without all this prior art already in place.