Building Software Packages for Linux <author> <tt><url url="mailto:thegrendel@theriver.com" name="Mendel Leo Cooper"></tt> <tt><htmlurl url="http://personal.riverusers.com/~thegrendel/" name="http://personal.riverusers.com/~thegrendel/"></tt> <date>v1.3, 13 August 1997 <abstract> This document is a guide to building "generic" UNIX software distributions under Linux. </abstract>  <toc>  <sect>Introduction Many software packages for the various flavors of UNIX, including Linux, are distributed as compressed archives of source files. The same package may be "built" to run an various target machines, and this saves the author of the software from having to produce multiple distributions. A single distribution of a software package may thus end up running, in various incarnations, on an Intel box, a DEC Alpha, a RISC workstation, or even a mainframe. Unfortunately, this puts the responsibility of actually "building" the software on the end user, the de facto "system administrator", the fellow sitting at the keyboard... you. Take heart, though, the process is not nearly as terrifying or mysterious as it seems, and this guide will attempt to demonstrate just that. <sect>Getting Started You have downloaded or otherwise retrieved a software package. Most likely it is archived (tarred) and compressed (gzipped), in <tt>.tar.gz</tt> or <tt>.tgz</tt> form. It must first be copied to a working directory. Then it is necessary to untar and gunzip it. The appropriate command is <bf>tar xzvf filename</bf>, where filename is the name of the software file, of course. The de-archiving process will usually install the appropriate files in subdirectories it will create. Note that if the package name has a <tt>.Z</tt> suffix, it will require <bf>uncompress PACKAGENAME</bf>, then <bf>tar xvf PACKAGENAME</bf> rather than the above procedure. Sometimes the archived file must be untarred and installed from the user's home directory, or perhaps in a certain other directory, according to the package's config info. Should you get an error message attempting to untar it, this may be the cause. Read the package docs (especially the <tt>README</tt> and/or <tt>Install</tt> files, if present) and edit the config files and/or <tt>Makefiles</tt> and <tt>Imake</tt> files, as necessary, consistent with the installation instructions. Some software packages permit automating this process by running <bf>make install</bf> to emplace the binaries in the appropriate system areas. You are now ready to proceed to the build stage of the process. <sect>Using Make The <tt>Makefile</tt> is the key to the build process. In its simplest form, a Makefile is a script for compiling or building the "binaries", the executable portions of a program. The Makefile can also provide a means of updating a software package without having to recompile every single source file in it, but that is a different story (or a different article). At some point, the Makefile launches <tt>cc</tt> or <tt>gcc</tt>. This is actually a preprocessor, a C (or C++) compiler, and a linker, invoked in that order. This process converts the source into the binaries, the actual executables. Only the simplest software uses a generic Makefile. More complex installations require tailoring the Makefile according to the location of libraries, include files, and resources on your particular machine. This is especially the case when the build needs the <tt>X11</tt> libraries to install. Imake and xmkmf accomplish this task. An Imakefile is, to quote the man page, a "template" Makefile. The imake utility constructs a Makefile appropriate for your system from the Imakefile. In almost all cases, however, you would run xmkmf, a shell script that invokes imake, a front end for it. Check the README or INSTALL file included in the software archive for specific instructions. Read the imake and xmkmf man pages for a more detailed analysis of the procedure.. Be aware that xmkmf and make may need to be invoked as root, especially when doing a <bf>make install</bf> to move the binaries over to the <tt>/usr/bin</tt> or <tt>/usr/local/bin</tt> directories. Using make as an ordinary user without root privileges will likely result in <tt>write access denied</tt> error messages because you lack write permission to system directories. Check also that the binaries created have the proper execute permissions for you and any other appropriate users. Invoking <bf>xmkmf</bf> uses the imake file to build a new Makefile appropriate for your system. It sets the variables and defines the library locations for the compiler and linker. Sometimes, there will be no imake file, instead there will be an INSTALL script that will accomplish this purpose. The README file included with the distribution will usually explain the install procedure. Your general installation procedure will therefore be: <itemize> <item>Read the README file. <item>Run <bf>xmkmf</bf> or the INSTALL script. <item>If necessary, run <bf>make clean</bf> or <bf>make depend</bf> <item>Run <bf>make</bf>. <item>If necessary, run <bf>make install</bf> </itemize> <sect>Troubleshooting If <bf>xmkmf</bf> and/or <bf>make</bf> succeeded without errors, you may proceed to the <ref id="finalsteps" name="next section">. However, in "real life", few things work right the first time. This is when your resourcefulness is put to the test. <sect1>Link Errors <itemize> <item>Suppose <bf>make</bf> fails with a <tt>Link error: -lX11: No such file or directory</tt>, even after <bf>xmkmf</bf> has been invoked. This may mean that the imake file was not set up properly. Check the first part of the Makefile for lines such as: <tscreen><verb> LIB= -L/usr/X11/lib INCLUDE= -I/usr/X11/include/X11 LIBS= -lX11 -lc -lm </verb></tscreen> The <tt>-L</tt> and <tt>-I</tt> switches tell the compiler and linker where to look for the library and include files, respectively. In this example, the X11 libraries should be in the <tt>/usr/X11/lib</tt> directory, and the X11 include files should be in the <tt>/usr/X11/include/X11</tt> directory. If this is incorrect for your machine, make the necessary changes to the Makefile and try the <bf>make</bf> again. <item>----------------------------------------------------------------- <item>In a very few cases, running <bf>ldconfig</bf> as root may be the solution: <bf># /etc/ldconfig -n /lib</bf> will update the shared library symbolic links. This should not be necessary under normal circumstances. <item>----------------------------------------------------------------- <item>Yet another thing to try if <bf>xmkmf</bf> fails is the following script: <tscreen><verb> make -DUseInstalled -I/usr/X386/lib/X11/config </verb></tscreen> <item>----------------------------------------------------------------- <item>Sometimes the source needs the older release X11R5 libraries to build. If you have the R5 libs in /usr/X11R6/lib (you were given the option of having them when first installing Linux), then you need only ensure that you have the links that the software needs to build. The <tt>R5 libs</tt> are named <tt>libX11.so.3.1.0</tt>, <tt>libXaw.so.3.1.0</tt>, and <tt>libXt.so.3.1.0</tt>. You generally need links, such as <bf>libX11.so.3 -> libX11.so.3.1.0</bf>. Possibly the software will also need a link of the form <bf>libX11.so -> libX11.so.3.1.0</bf>. Of course, to create a "missing" link, use the command <bf>ln -s libX11.so.3.1.0 libX11.so</bf>, as root. </itemize> <sect1>Other Problems <itemize> <item>An installed Perl or shell script gives you a <tt>No such file or directory</tt> error message. In this case, check the file permissions to make sure the file is executable and check the file header to ascertain whether the shell or program invoked by the script is in the place specified. For example, the scrip may begin with: <tscreen><verb> #!/usr/local/bin/Perl </verb></tscreen> If Perl is in fact installed in your <tt>/usr/bin</tt> directory instead of the <tt>/usr/local/bin</tt> one, then the script will not run. Edit and correct the script file header in such a case. <item>----------------------------------------------------------------- <item>Some X11 software requires the <tt>Motif</tt> libraries to build. The standard Linux distributions do not have the Motif libraries installed, and at present Motif costs an extra $100-$200 (though the freeware <tt>Lesstif</tt> may also work in many cases). If you need Motif to build a certain package, but lack the Motif libraries, it may be possible to obtain statically linked binaries. Static linking incorporates the library routines in the binaries themselves. This results in much larger binary files, but the code will run even on systems lacking the libraries. </itemize> <sect1>Where to go for more help In my experience, about 25% of applications build "right out of the box". Another 50% or so can be persuaded to build with an effort ranging from trivial to herculean. That still means a significant number of packages will not build no matter what. Even then, the Intel <tt>ELF</tt> and/or <tt>a.out</tt> binaries for these might possibly be found at <url url="ftp://sunsite.unc.edu" name="Sunsite">, the <url url="ftp://tsx-11.mit.edu" name = "TSX-11 archive"> or other places. Perhaps the author of the software can supply the binaries compiled for your particular flavor of machine. <tt>Note that if you obtain precompiled binaries, you will need to check for compatibility with your system:</tt> <itemize> <item><tt>The binaries must run on your hardware (i.e., Intel x86).</tt> <item><tt>The binaries must be compatible with your kernel (i.e., a.out or ELF).</tt> </itemize> If all else fails, you may find help in the appropriate newsgroups, such as <htmlurl url="news://comp.os.linux.x" name="comp.os.linux.x"> or <htmlurl url="news://comp.os.linux.development" name="comp.os.linux.development">. Once in a while, though, you are just plain out of luck, but hey, it was fun trying. <sect>Final Steps<label id="finalsteps"> Read the software package documentation to determine whether certain environmental variables need setting (in .bashrc or .cshrc) and if the .Xdefaults and .Xresources files need customizing. There may be an applications default file, usually named Xfoo.ad in the original Xfoo distribution. If so, edit the Xfoo.ad file to customize it for your machine, then rename (<bf>mv</bf>) it Xfoo and install it in the <tt>/usr/lib/X11/app-defaults</tt> directory, as root. Failure to do this may cause the software to misbehave or even refuse to run. Most software packages come with one or more preformatted man pages. As root, copy the Xfoo.man file to the appropriate <tt>/usr/man</tt> directory (<tt>man1</tt> - <tt>man9</tt>), and rename it accordingly. For example, if Xfoo.man ends up in /usr/man/man4, it should be renamed Xfoo.4 (mv Xfoo.man Xfoo.4). Note that some or all of the above procedures may in certain cases be handled automatically by a <bf>make install</bf>. If so, the <tt>README</tt> or <tt>INSTALL</tt> doc file will specify this. <sect>First Example: Xscrabble Matt Chapman's <bf>Xscrabble</bf> seemed like a program that would be interesting to have, since I happen to be an avid Scrabble<tt>&tm;</tt> player. I downloaded it, uncompressed it, and built it following the procedure in the README file: <tscreen><verb> xmkmf make Makefiles make includes make </verb></tscreen> Of course it did not work... <code> gcc -o xscrab -O2 -O -L/usr/X11R6/lib init.o xinit.o misc.o moves.o cmove.o main.o xutils.o mess.o popup.o widgets.o display.o user.o CircPerc.o -lXaw -lXmu -lXExExt -lXext -lX11 -lXt -lSM -lICE -lXExExt -lXext -lX11 -lXpm -L../Xc -lXc BarGraf.o(.text+0xe7): undefined reference to `XtAddConverter' BarGraf.o(.text+0x29a): undefined reference to `XSetClipMask' BarGraf.o(.text+0x2ff): undefined reference to `XSetClipRectangles' BarGraf.o(.text+0x375): undefined reference to `XDrawString' BarGraf.o(.text+0x3e7): undefined reference to `XDrawLine' etc. etc. etc... </code> I enquired about this in the <htmlurl url="news://comp.os.linux.x" name="comp.os.linux.x"> newsgroup, and someone kindly pointed out to me that apparently the Xt, Xaw, Xmu, and X11 libs were not being found at the link stage. Hmmm... There were two main Makefiles, and the one in the src directory caught my interest. One line in the Makefile defined LOCAL_LIBS as: LOCAL_LIBS = $(XAWLIB) $(XMULIB) $(XTOOLLIB) $(XLIB) Here were references to the libs not being found by the linker. Looking for the next reference to LOCAL_LIBS, I saw on line 495 of that Makefile: <tscreen><verb> $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LOCAL_LIBS) $(LDLIBS) $(EXTRA_LOAD_FLAGS) </verb></tscreen> Now what were these LDLIBS? <tscreen><verb> LDLIBS = $(LDPOSTLIB) $(THREADS_LIBS) $(SYS_LIBRARIES) $(EXTRA_LIBRARIES) </verb></tscreen> The SYS_LIBRARIES were: <tscreen><verb> SYS_LIBRARIES = -lXpm -L../Xc -lXc </verb></tscreen> Yes! Here were the missing libraries. Possibly the linker needed to see the LDLIBS before the LOCAL_LIBS... So, the first thing to try was to modify the Makefile by transposing the $(LOCAL_LIBS) and $(LDLIBS) on line 495, so it would now read: <tscreen><verb> $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LDLIBS) $(LOCAL_LIBS) $(EXTRA_LOAD_FLAGS) ^^^^^^^^^^^^^^^^^^^^^^^ </verb></tscreen> I tried running <bf>make</bf> again with the above change, and lo and behold, it worked this time. Of course, Xscrabble still needed some fine tuning and twiddling, such as renaming the dictionary and commenting out some assert statements in one of the source files, but since then it has been providing me with many hours of pleasure. You may e-mail <url url="mailto:matt@belgarath.demon.co.uk" name="Matt Chapman">, and download <bf>Xscrabble</bf> from his <url url="http://www.belgarath.demon.co.uk/programs/index.html" name="home page">. <code> Scrabble is a registered trademark of the Milton Bradley Co., Inc. </code> <sect>Second Example: Xloadimage The second example poses an easier problem. The <bf>xloadimage</bf> program seemed a useful addition to my set of graphic tools. I copied the xloadi41.gz file directly from the source directory on the CD included with the excellent <ref id="refs" name="X User Tools"> book, by Mui and Quercia. As expected, <bf>tar xzvf</bf> unarchives the files. <bf>Make</bf>, however, produces a nasty-looking error and terminates. <code> gcc -c -O -fstrength-reduce -finline-functions -fforce-mem -fforce-addr -DSYSV -I/usr/X11R6/include -DSYSPATHFILE=\"/usr/lib/X11/Xloadimage\" mcidas.c In file included from /usr/include/stdlib.h:32, from image.h:23, from xloadimage.h:15, from mcidas.c:7: /usr/lib/gcc-lib/i486-linux/2.6.3/include/stddef.h:215: conflicting types for `wchar_t' /usr/X11R6/include/X11/Xlib.h:74: previous declaration of `wchar_t' make[1]: *** [mcidas.o] Error 1 make[1]: Leaving directory `/home/thegrendel/tst/xloadimage.4.1' make: *** [default] Error 2 </code> The error message contains the essential clue. Looking at the file image.h, line 23... <code> #include <stdlib.h> </code> Aha, somewhere in the source for xloadimage, <tt>wchar_t</tt> has been redefined from what was specified in the standard include file, <tt>stdlib.h</tt>. Let us first try commenting out line 23 in <tt>image.h</tt>, as perhaps the <tt>stdlib.h include</tt> is not, after all, necessary. At this point, the build proceeds without any fatal errors. The <bf>xloadimage</bf> program functions correctly now. <sect>Final Words To sum up, persistence makes all the difference (and a high frustration threshold certainly helps). As in all endeavors, learning from mistakes is critically important. Each misstep, every failure contributes to the body of knowledge that will lead to mastery of <bf>the art of building software</bf>. <sect>References and Further Reading<label id="refs"> <tscreen><verb> BORLAND C++ TOOLS AND UTILITIES GUIDE, Borland International, 1992, pp. 9-42. [One of the manuals distributed with Borland C++, ver. 3.1. Gives a fairly good intro to Make syntax and concepts, using Borland's limited implementation thereof.] DuBois, Paul: SOFTWARE PORTABILITY WITH IMAKE, O'Reilly and Associates, 1996, ISBN 1-56592-226-3. [This is reputed to be the definitive Imake reference, though I did not have it available when writing this article.] Lehey, Greg: PORTING UNIX SOFTWARE, O'Reilly and Associates, 1995, ISBN 1-56592-126-7. Mui, Linda and Valerie Quercia: X USER TOOLS, O'Reilly and Associates, 1994, ISBN 1-56592-019-8, pp. 734-760. Oram, Andrew and Steve Talbott: MANAGING PROJECTS WITH MAKE, O'Reilly and Associates, 1991, ISBN 0-937175-90-0. Stallman, Richard M. and Roland McGrath: GNU MAKE, Free Software Foundation, 1995, ISBN 1-882114-78-7. Welsh, Matt and Lar Kaufman: RUNNING LINUX, O'Reilly and Associates, 1995, ISBN 1-56592-100-3, pp. 325-333, 377-379, 381. [Still the best overall Linux reference, though lacking in depth in some areas.] </verb></tscreen> And, of course, the <tt>man pages</tt> for make, imake, xmkmf, and ldconfig. </article>