Shared libraries in Linux: growing pains or fundamental problem?
Introduction: is system DLL problem Windows-specific?
The speed of development of Linux is usually considered a very good
thing. Now I shall talk about one aspect of it, namely quick change
of software components commonly known as shared libraries, or DLLs
(dynamically linked libraries). This is often thought to be serious
problem in the Windows world, when some new update which comes with
new version of some product will change system-wide libraries, and
as a result some other software packages would not run properly
(DirectX is particularly notorious for that effect; it is not that
surprising given that some parts of it are hardware-specific).
However, this problem is also present in Linux world, and in fact
it might be even worse there. The idea to write this article has
emerged after rather long talk with my friend who
was telling me about his
RedHat 6.0 experience. You will find some
extracts from our talk in the text below.
Upgrading system: from libc 5 to glibc 2.0 to glibc 2.1
The first and rather obvious problem about shared libraries and their
versions is that all software you had linked against old library have
to be recompiled.
This is not always the case; after all, the versions
of glibc are not designed to be incompatible with each other; however,
little things have crept here and there, and not all programs built
with
glibc 2.0.x
will run with glibc 2.1. But when upgrading from libc5
to glibc, this is a necessity -- programs linked with libc5 will not run
with glibc2 and either have to be recompiled or you'll have to keep
libc5 libraries around just to be able to run binaries linked with them.
With very complex software packages like
GNOME or
KDE this becomes real problem: it is
often easier just to download recompiled binaries than to rebuild
them from the sources or make older binaries work on a new system --
of course, if new binaries are available! The prerequisite list
of required libs for monsters like GNOME is quite long (13 pieces),
and it is very easy to break the entire package just by installing
newer/older version of libpng.
Linux distributions
This binary compatibility effect also has great impact on Linux
distributions. Namely, if you don't want to spend the rest of your
life recompiling things, you often have to stick to very limited
set of distributions. Not all distros include everything, and sticking
RPMs from RedHat into
Slackware usually only works for
more or less
simple software. With complex things, lots of library dependencies make
using foreign binaries unrealistic.
My friend explained to me how he
managed to install Russian fonts into Ghostscript on his RedHat. He
simply took them from
KSI Linux (this is a Russian Linux distribution).
I have asked him why did he not try the KSI Linux itself. He told me that
he wanted to stay mainstream, just to make sure that everything made
for Linux will run on his system without recompile.
Essentially, this makes smaller distributions (like KSI) to be pushed
off scene: users want to be compatible! My friend was especially
concerned about
StarOffice and
IBM DB2.
Since both software packages were not tested on KSI by their manufacturers,
will they run on KSI or not is not
something he was willing to investigate. This leads us to next major
obstacle: running closed-source software.
Proprietary software
Whatever some people might say, proprietary (i.e. closed-source) software
will continue to exist in foreseeable future. Recently we had a whole
lot of news on this arena: from big database vendors (Oracle, IBM DB2,
Informix) to games (Quake,
Civilization). Apparently it is quite
unlikely that companies which consider their technologies and art top
secret are going to release the source any time soon. Therefore we have
to put up with binary only releases. With time, the situation might
get better, but it is as likely will get worse as new software
players will enter growing Linux market. The aforementioned problem
is very serious with this kind of software. When you cannot recompile
the thing, you'll have to stick to what it will run with -- in other
word, you are some sort of locked into having predefined set of shared
libraries in your system! Here comes another little story: my
friend who has just installed shiny new RedHat 6.0 was very enthusiastic
about it, except one thing: Star Office would not run! Software wanted
glibc 2.0.7 while RedHat has switched to glibc 2.1. I heard (please don't
blame me if it is wrong) that initial version of IBM ViaVoice
for Linux was linked against glibc 2.1 (on RedHat 6.0). This essentially
means that StarOffice and ViaVoice cannot run at the same time.
The rest of the world
If you will read
FreeBSD mailing lists,
you'll discover a rather
abundant category of questions: how to run some Linux binary-only
package on FreeBSD. Since FreeBSD has a Linux emulation layer, it
is often not a problem to use Linux-specific software on FreeBSD.
But sometimes it is -- when the Linux binary depends on several
libraries of particular versions. It is interesting to note that on
the FreeBSD itself the library compatibility problem is less important:
system core just does not change frequently enough. The major FreeBSD
releases happen much less often than Linux ones, and there are no
concerns about distributions because there's only one.
The situation with X11 libraries is quite similar to FreeBSD's one.
Since major revisions of the X11 take years, the entire set of calls
and DLLs remains very compatible between versions. Some people would
say however that X needs serious overhaul because it is outdated :-)
At the time of the writing, next revision of
XFree86 (4.0) is not yet released.
Possible solutions
What possible solutions could be offered for this problem? Honestly,
I am not an OS developer, and my point of view is not probably well
thought out. I can think of several ways of resolving the difficulty:
- use statically linked executables. This approach is generally
regarded as undesirable because it kills all advantages of
dynamic linking. It causes major bloat; programs
tend to consume much more disk space and RAM when linked statically.
If the library does contain a bug, it will be linked hard into the
executable and cannot be fixed by replacing shared library.
Sometimes, however, this is the best and simplest way to achieve binary
compatibility. With closed source software, statically linked binaries
should be provided as a fallback, together with dynamically linked ones.
In most other cases, their use should be discouraged.
- switch to finer version numbering. In other words, let to link
to libc-2.0.so or even libc-2.0.7.so instead of libc.so.6. Apparently
pursuing this approach too far is undesirable as libc-2.0.7 is supposed
to hold latest bugfixed in libc-2.0.x line; the limit has to be put
somewhere. But in ultimate cases, it could make sense to link even with
specific versions of a library because really complex software could
be quite touchy about library version. Correct me if I am wrong, but
in current incarnation everything is supposed to be linked against
libc.so.6 (with glibc 2.x).
- expand the definition of Linux kernel to include at least libc and
several other basic libraries (termcap, ncurses, pthreads?) and
make those libraries stable. This
has many rather obvious drawbacks: licensing issues, does not solve
all of the problem as kernel releases are also happen quite often,
nobody can hope to include everything (GTK?). Some things just can't
be stabilized because they are under active development.
Besides, the current
trend has exactly opposite direction; in most recent Linux distribution
libc5 (which was being viewed as Linux-only) is replaced with
glibc2 (which is supposed to be the ultimate libc on Earth).
- create some sort of a committee which will make prepackaged
version of standard libraries (both in source and binary form),
and use of library versions which are not in the packages will be
highly discouraged. The version numbers will be assigned to the
packages as a whole, and the rate of package major releases (which
are not 100% compatible with each other) should be much slower
(say, once a year). This is the technique used in FreeBSD distributions,
X11, Qt and many others. The drawbacks: lots of political issues,
slow rate of new technology adoption. Some people with too much love
for creeping featurism might seem this as a
loss of their freedom because committee will have to make hard
decisions from time to time.
In some cases, the solution will involve a lot of political struggle
and debates. But whether the threat
of binary incompatibility is real or it is just a growing pains
of a young operating system remains to be seen...
Disclaimer
I am not Linux core developer. I do
write software on Linux and other
OSes, but my understanding of some basic issues might be wrong in
this writing. I will readily accept any corrections for mistakes.
Links to related materials are also welcome.
Further reading:
GNU libc-6 announcement.
Copyright (C) 1999 Sergey Ayukov. Parts of this text can be reproduced
without permission, with reference to this page or the author.
16 May 1999
return to essays |
return to homepage |
send comment