Inter-library Dependencies

About twice a week, for the last five weeks, I've been receiving bug reports which tell me that libtool's inter-library dependency handling is broken.

I know. I broke it intentionally, until I have the time to fix it myself, or somebody else takes the time to help me with it.

These same people often give me a simple one-line patch which re-enables my old, simplistic inter-library dependencies, but nobody seems to want to test things thoroughly and come up with a real solution.

If you don't care about the history, and you just want to help me out, jump to the bottom of this document.

Background

Libtool's basic premise is to make static and shared libraries behave the same way from a programmer's point of view. This allows users to build a libtoolized package with or without shared libraries, determined at configuration time. It does this by using a libtool object (.lo) and libtool archive (.la) abstraction, so that the package maintainer can use libtool to operate on these files without making any assumptions about their underlying representation.

For the most part, this abstraction works well, and has made libtool as popular as it is today. Without this abstraction, it would be significantly harder to port libtool to new platforms.

Unfortunately, what this abstraction has also done is reveal some fundamental inconsistencies with most shared library implementations. Every shared library implementation works well for `hello world'-type examples, but very few are robust and well-designed so that libtool doesn't need special tricks in order to build correct, featureful shared libraries.

Providing inter-library dependencies is one feature that has revealed these kinds of inconsistencies.

The problem

My original inter-library dependency code received rigourous testing in beta releases of GNU Guile. As soon as the Guile people started using my code, I received a flood of bug reports. People were reporting that libguile (Guile's main shared library) was failing to link, or that programs linked against libguile were dumping core.

Not good.

The Guile people chased this bug down to the following scenario:

1. The user's system has a static regexp library installed, librx.a.
2. Guile's configure script detects that the -lrx linker option can be used to link in librx.
3. When libguile is built, the -lrx linker option is used.
4. Some linkers fail at this point, because they don't allow shared libraries to contain or depend on static libraries.
5. If the linker didn't fail, then a few programs are linked against libguile.
6. On some systems, these programs core dump because libguile is a shared library that contains non-PIC code (from librx).

The interim solution

I needed some way to respond to these reports. I saw my options (in order of my preference) as:

1. Write code in ltmain.sh to prevent static libraries from appearing in inter-library dependencies. This would take some work, but obviously is the best solution.
2. Find the systems that fail, and turn off inter-library dependencies on only those systems.
3. Force the package maintainer to guarantee that static libraries never appear in inter-library dependencies.

I immediately vetoed the last solution, because that would violate the whole point of using libtool, and would cause a lot of people to waste time solving a problem that really should be fixed by libtool.

I preferred the first solution, but at the time of the reports, I didn't see an obviously simple mechanism for detecting the difference between shared and static libraries.

So, in the meantime, I tried turning off inter-library dependencies on the systems that failed.

I quickly discovered, to my chagrin, that many systems fail. So, it was be simpler for me to turn off all inter-library dependencies, then find out which systems work, rather than vice versa.

The current situation

I've been busy trying to avoid bankruptcy. It's been over three months since I first turned off inter-library dependencies, and I still haven't completed the solution I want.

I've been gearing up for the 1.1 release of libtool, because there is a high demand for a stable public release. So, I'm not going to introduce any destabilizing changes to the inter-library dependency code until after 1.1 is released.

The solution

So, I want to tell you how you can help me solve the various dilemmas surrounding this issue:

1. I need to find out more about the nature of the problems I ran into with Guile. Unfortunately, I cannot reproduce them in simple tests on my own platform (i586-pc-linux-gnulibc1), even though I think they were reported here. I need to find out which platforms already have perfect inter-library dependency support, how to work around the problem on other platforms, and, more importantly, exactly why some systems give me problems and others don't.

   On correct platforms, you can link any static library against a shared library via the -lNAME option without the linker complaining, then link a program against this library and run it without dumping core. I know that this scenario will always work fine on the following systems:

   • None reported yet.

   I also know that on some systems, you can create a shared library linked against a static one, but running programs linked against such a library will dump core:

   • None reported yet.

   Finally, there are some systems which won't even allow you to link a shared library against a static one:

   • Solaris 2.x
2. Help me figure out a good, portable way to detect if a given -lNAME option refers to a shared library or not, since that is needed as a workaround to the problem. Some suggestions so far have been:
   • Link the library against a tiny test program, and:
     • run ldd(1) on the test program and search for libNAME in the ldd output.
     • use the -verbose flag for GNU ld in order to see which library is actually linked.
     • run some sort of other program to determine if the library was dynamic.
   • Track the -LDIR flags, do a search for the library, and then check whether it is shared by:
     • using the file(1) program.
     • looking at its suffix.
3. Contact any people you know who might be interested, get them to read this page, so that they can help me solve the problem.
4. Send me money, so that I can devote more of my time to solving the problem.

Thank you for your help, and have fun.

Back to the libtool home page.