BeleniX packaging and repository directions part 1

There has been quite a bit of thinking and activity going around with the long term plan for package and repository management around BeleniX. This is an urgent issue requiring a working implementation “today” and also a proper long-term implementation. We have had extensive in fact excrutiating discussions and head-banging reagarding this and have finally come up with a plan. We are getting a temporary solution ready which will serve the needs probably for the next 6 months giving us breathing time to move to a more permanent and robust solution. Before I jump in with our observations and conclusions lets look at some of the requirements. These are mostly common with what any typical Linux distro will need today:

  1. Easily accessible network repository providing a large repertoire of packages. This probably the single largest thing of value to a distribution. No one is going to use a distro that does not have a large collection of software going for it.
  2. Ability to support package repository mirrors in general dumb Apache based download mirrors. This point is important in getting to leverage public mirror services. So dumb metadata servers and rich clients.
  3. Ability to distribute local copies of packages on media like DVD or USB pendrive and use these as local non-networked repository sources. This is quite important in our view since a vast portion of the world’s computer users do not have access to always-on high-speed broadband network connection – the network is just not ubiquitious, not yet.
  4. Use of maximum compression to minize network transfer requirements.
  5. Automatic dependency resolution with sane heuristics.
  6. Advanced searching and boolean dependency specifications.
  7. Multi-repository support (slightly different from mirrors).
  8. Safe upgrades (downgrades if possible) leveraging ZFS.
  9. Release based tagging of repository allowing upgrades to intermediate releases.
  10. Support OpenSolaris Zones.
  11. Easy to operate and simple for contributors.
  12. Leave the build system out and no support for patching madness. It is just package revisions.
  13. Good web interface and high-performance client-side operations.
  14. Leverage years of community work in these areas instead of re-inventing the wheel.

As you can see most of the requirements are typical. BeleniX at present still uses the old SVR4 packaging system that has been the native packaging for Solaris/OpenSolaris till recently. SVR4 has a bunch of problems and does not satisfy most of the requirements. Now we were faced with a predicament. We need these things right away but changing from SVR4 to a different packaging strategy involves a lot of effort around distro tooling and testing that will take time. However we have the next BeleniX release 0.7.2 in the queue with 0.8 afterwards with KDE4 and all requiring a lot of work.

Stuck between a rock and hard place, we were looking for a temporary solution around SVR4 to buy us time. We were initially looking at pkg-get of Blastwave fame. Unfortunately that turns out to not be Free and Open Source. It has a restrictive licensing put by it’s author. So we were stuck!

Finally I decided to stop searching and start coding. That is I decided to write a utility from scratch that will layer over SVR4 base packaging and provide all the goodies. It will provide a combination of features from pkg-get and IPS and I decided to make it Pythonic since that is a language I am familiar with and provides benefits over a simple shell scripts. I have been working on it for a couple of weeks and already have most of the implementation done including core framework, install, multi-repository handling etc. I am presently working on upgrade and query. You can see the code here:

http://belenix.svn.sourceforge.net/viewvc/belenix/trunk/spec_files/ext-sources/spkg_mod.py?revision=153&view=markup

http://belenix.svn.sourceforge.net/viewvc/belenix/trunk/spec_files/ext-sources/spkg?revision=151&view=markup

http://belenix.svn.sourceforge.net/viewvc/belenix/trunk/spec_files/ext-sources/spkg.conf?revision=153&view=markup

It appears that I will be able to finish this work in another week or so. Yes I am re-inventing the wheel partly but out of necessity. In part 2 of this blog series I will look at detailes of the implementation and repository and release management structure.

Now we had lot of discussions around our long-term approach and we looked at several solutions including of course IPS. Our driving factor is to stay aligned with the OpenSolaris community and not diverge too much and from that viewpoint IPS should have been the logical choice. However our final decision is to stay away from IPS. There are a whole bunch of considerations that went into that decision. See this discussion thread.

We are looking at multiple possibilities including using Dpkg based on Nexenta’s efforts on making it work on OpenSolaris as the native packaging mechanism. However instead of APT¬† it is possible to use Smart. I have been quite excited reading about the advanced capabilities in Smart. Though this is still being considered and we have 6 months to think about it.

I will detail our current repository structure and ‘spkg’ semantics in the next post.

Advertisements

2 thoughts on “BeleniX packaging and repository directions part 1

  1. Shiv

    The points on pkg-discuss captures all the points that have been discussed. It is a shame that we cannot go with the default package management system of Indiana but then we do not have a choice due to the reasons listed.

    Reply
  2. Sriram "Ram" Narayanan

    Based on the inputs given by various members of the IPS team, I am optimistic that they will supply the promised feature set one day, and also address the various issues that we’ve highlighted.

    However, Belenix needs a packaging scheme _now_, and I give my own +1 to what Moinak has designed for now. Once the packages are in place, we can indeed start to look at Smart and add to it those features of IPS which Smart lacks.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s