Contact SalesSitemapCustomer Login

Custom Linux Environment - Part 01 - Compilers

Contents

[edit]

Prologue

First off, I'd like to point out a few obvious things. Slackware's learning curve is listed as Steep. System Administration is complex and, ultimately, incredibly customizable. There are dozens of ways to do a given task, and I'm not going to start to cover them all. I'm only going to cover My Way.

This article's topic is, essentially, how to install everything that's needed to compile things. We will do a lot of work by hand so be prepared. If you want an easy guide with very few commands, please check out a distribution that has yum or apt-get. In Slackware, even package installation is a very manual task: There is no double-checking the user, there is no dependency checking. Slackware does what you tell it to -- no more, no less.

This is also an educational article. I'm going to walk you through Troubleshooting Logic 101. It boils down to this rule of thumb: Test. Attempt to fix the first and only the first error. Test again. Repeat until there are no errors left. It works in programming, in web development, in systems administration and in interoperability tests.

[edit]

Initial Setup

Before we can begin we need a basic set of compilers. Most of the other things needed we can build by hand. The packages we are going to install can always be replaced at a later date. Ultimately, this is what is classified as a quick and dirty guide.

[edit]

GCC

First we need GCC. In a Link1 package the environment is very limited. I am currently doing this on a Link1 account. We have no wget, no editor but vi (which I don't use) and no lynx text web browser. We have to do everything by hand.

We do have one program we can use to download: Good old FTP.

[edit]

Obtaining the package

Go to www.slackware.com and find yourself a good mirror; whichever one you like. Then find the pub/slackware/slackware-12.1/slackware/d/ in it. You will find a number of files, but the one we want is gcc-4.2.3-i486-1.tgz. Get it.

Your user name should be "anonymous" and your password should be an email address that can be used to reach you, according to ye olde traditions.

[edit]

Installing the package.

The command you want in slackware, to install the package you just obtained by hand, is:

installpkg gcc-4.2.3-i486-1.tgz

You should see a block of text like this:

Installing package gcc-4.2.3-i486-1... 
PACKAGE DESCRIPTION:
gcc: gcc (Base GCC package with C support)
gcc:
gcc: GCC is the GNU Compiler Collection.
gcc:
gcc: This package contains those parts of the compiler collection needed to
gcc: compile C code.  Other packages add C++, Fortran, Objective-C, and
gcc: Java support to the compiler core.
gcc:
Executing install script for gcc-4.2.3-i486-1...

[edit]

Testing the package.

Testing is simple. Just run this command:

gcc --version

You should get the following output:

gcc (GCC) 4.2.3
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If you get it, great.

If not, then removepkg /var/log/packages/gcc* and then rm gcc* and download it again because you got a corrupted copy.

If it doesn't work three times in a row, use your control panel to reinstall slackware 12.1.

[edit]

Compiling a test program

Next, we want to make sure we can compile. (I can't, but what is true on my Link1 package may not be true on yours.)

"But," I hear you ask, "How can we make a test program to compile with out an editor?"

Grab a cup of coffee and observe. Copy this and paste it into your terminal window:

cat - >> test.c << EOF
#include <stdio.h>
int main()
{
   printf("Goodbye, cold, distant, cruel compiler! Hello, new, working compiler friend!\n");
   return(0);
}
EOF

If you don't believe this actually worked, you can test it. Just enter "cat test.c".

To compile it the command is:

gcc test.c -o test

Here is the output I get on my Link1 account:

test.c:1:19: error: stdio.h: No such file or directory
test.c: In function 'main':
test.c:4: warning: incompatible implicit declaration of built-in function 'printf'
test.c:4: error: stray '\' in program
test.c:4: error: expected ')' before 'n'

What does this mean? Well, rhetorical question time, folks. Get yourself a cup of coffee and prepare for an explanation SO LONG you'll have time to take half a sip.

stdio.h doesn't exist in /usr/include. In fact, if you ls /usr/include, you'll find it is pretty empty. Well, entirely empty, actually. This means that we don't have the full glibc installed. We need the full glibc. Unfortunately, that means we're going to have to overwrite the libc.so.6 file that Every important program you have uses. To reiterate, if this fails you will be left without a working system.

So, drink the rest of that cup of coffee, and think really, really hard - Is there anything I'd be upset if I lost? If so, and I can't stress this enough, BACK IT UP.

[edit]

GLIBC reinstall

[edit]

Download the tarball.

We now need to obtain the glibc package. This will blow about 28MB of bandwidth. That is real, binary megabytes, not "Defined as 30000000 kilobytes." Now, obviously, we have the solibs glibc package from the A package of slackware. How do I know? We can log in. Go ahead, double check - ldd /usr/sbin/sshd - You'll see a link to libc.so.6.

What we need is the rest of the glibc package. Log back in to your chosen and preferred slackware mirror, and go into /pub/slackware/slackware-12.1/slackware/l/ - Snag the glibc-2.7-i486-10.tgz file. Installpkg it. You'll see it even tells us, "You'll need this to compile programs."

[edit]

Testing GCC again

So, we gcc test.c -o test again. Let's see what we get:

gcc: error trying to exec 'as': execvp: No such file or directory

This is where experience and knowledge begin to play a role in systems administration. Error trying to execvp 'as'? What is 'as'? What is 'as'. . . what. . . huh 'as' who? Have a drink of coffee. 'as' is the GNU Assembler. We have completed the first step of compiling test.c successfully. Now we have a new error to address.

[edit]

Obtaining the mysterious 'as'

This time we are back to our familiar D section of slackware packages. If you do not know, D stands for Development, A for Absolutely Required, L for Library, N for Network, AP for APplications. Most of them are fairly easy to remember.

The GNU Assembler is part of the GNU Binary Utilities, also known as binutils.

[edit]

Sublesson in environment manipulation: Alias

I don't like typing in ftp my.favorite.slackware.mirror all the time and then logging in. So, I use the alias command. It works like this:

alias mir="ftp my.favorite.slackware.mirror"

Now simply type mir (or whatever shorthand reference you chose) at your prompt and it is the same as typing ftp my.favorite.slackware.mirror.com. I find this very useful.

[edit]

Getting the package

It's called binutils. It's in the D section. Go download it.

Installpkg it.

[edit]

Testing GCC Again

Like the subsection title says, test it!

gcc test.c -o test

If you have not gotten output, there is something wrong. ls and look around. Is there an error file?

What is this green test thing? Oh wait, the only problem is we have nothing to fix. Whoops. Try running the test file:

./test

You should get some output marking the changing of a virtual, very short era:

Goodbye, cold, distant, cruel compiler! Hello, new, working compiler friend!

Congratulations. You now have a compiler that works on the most basic level. But we want something that works completely, so let's test it out. In fact, I don't have an editor I like yet so I am going to use Joe. You can get it from Joe's site, but if you take a good look at all the mirrors, you'll find they're all http. We can't download from http. The current version is 3.7 and slackware only has version 3.5. Since we like shiny and new, (or we wouldn't be compiling in the first place) we need a program called wget.

[edit]

Getting Wget

Now, wget is GNU software. That means we can get the source from GNU, and they have FTP mirrors. In fact, they have ftpmirror.gnu.org as a target that automatically directs you to the closest mirror if you have http. So, without further ado, go here and find yourself a good mirror. ftp into it. Snag wget-1.11.4.tar.bz2 (It is the newest version at the time of this writing.) and exit FTP.

[edit]

Decompressing wget

It's filename ends with .bz2. This means it is a bzip2 archive. This is the command I use to decompress bz2 files.

(For those of you who are curious about my strange syntax, you have to understand, I've been around since before tar even began to understand gz or bz2 compression natively.)

bzip2 -cd wget-1.11.4.tar.bz2 | tar xf -

If you want to see a list of files decompressed, use tar xvf -.

cd into your new wget directory. There is a very simple normal progression in compiling most software from source: Configure, make, make check or test, make install.

[edit]

Configuring wget

It is pretty simple. Just do this:

./configure

But /lib/cpp fails sanity check!? That is insane! We just tested the compiler! Check out the config.log file the output tells us to look at for more information - see if you can figure out on your own what is wrong before continuing.

You will find this little gem:

/usr/include/bits/local_lim.h:36:26: error: linux/limits.h: No such file or directory

Let's take a peek at /usr/include/linux, where limits.h is supposed to live.

ls /usr/include/linux

As you can see, there is a problem. If you do not run that command yourself you will not know why I say that. Obviously, we are supposed to have a /usr/include/linux/limits.h, but we do not. How do we get it?

[edit]

Sanitizing Linux Kernel Headers

Note:

You very rarely want to do this. You will do this on a system on which there is no other choice or when you are compiling a new glibc by hand. These are the only times you will ever do this.


We need sanitized kernel headers. The kernel headers, when sanitized, provide what is called a stable ABI. ABI stands for Application Binary Interface. You see, the ABI (and the API) in the linux kernel changes frequently. A long time ago, we just had a symlink at /usr/include/linux which pointed to /usr/src/linux/include/linux, which was in turn symlinked to /usr/src/linux-1.1.19 or whatever we had installed. That technique died because ABI (and API's) in the linux kernel can change between 2.6.18.8 and 2.6.19.0 with no warning whatsoever to most folk.

But for newer linux kernels you cannot get sanitized kernel headers anymore. You can as of linux kernel 2.6.18 sanitize your own headers. This is a godsend to many of us who do crazy things, like compile glibc by hand.

[edit]
Downloading the linux kernel headers

For this one do not use a mirror. Just ftp kernel.org and go into /pub/linux/kernel/v2.6; get linux-2.6.18.tar.bz2. You will blow another, oh, about 37MB for this one. It is fine because even on a Link1 you have 100GB and we will probably not blow 700MB total doing it twice.

[edit]
Creating sanitized headers
bzip2 -cd linux-2.6.18.tar.bz2 | tar xf -

Now cd into it and:

make mrproper

Since it is a good idea to make sure everything is proper, we always start off with Mr. Proper when we are dealing with a brand new linux kernel source tree. However, we get an error this time. We do not actually have make. We need make. All right.

[edit]
Obtaining Make

You can get the newest version of make (3.81 at the time of writing) off of your favorite GNU mirror, directory /gnu/make or /pub/gnu/make. Snag it now.

[edit]
= The chicken and egg problem =

Next we go into the tree and see a program called "configure" when we ls. This is bad. We need make to make make? How can we get make if we don't already have it? This is an obvious example of a chicken and egg problem. However, GNU is populated by very bright people, and they have packaged a solution.

Many times, you will run into chicken and egg problems where you have to compile a partial version of one program to compile a full version of a second program, just so you can go back and compile the full version of the first. Interdependencies can be incredibly frustrating. There is a reason Pat, the maintainer of slackware, has stated that compiling GNOME by hand is the equivalent of herding angry cats.

Here's a quote from the README file:

If you need to build GNU Make and have no other `make' program to use,
you can use the shell script `build.sh' instead.  To do this, first run
`configure' as described in INSTALL.  Then, instead of typing `make' to
build the program, type `sh build.sh'.  This should compile the program
in the current directory.  Then you will have a Make program that you can
use for `./make install', or whatever else.

In other words, it gives us a basic make we can use to make the real make. In slackware linux, reading documentation is important. If you have no familiarity with technical documentation, you can probably go to your local college or university and audit a class on technical documentation. Auditing lets you take the class for a small fee, because you do not get any credits or an entry into your transcript.

Go ahead and ./configure it!

Welcome to a real chicken and egg problem, brought on by our situation. There is not any solution we can do on our own so we have too choose between two options. We can either install the precompiled make, or we can install the linux kernel headers.

Let's check the linux kernel headers first. They are in the D package of your favorite slackware FTP mirror.

Look at the version number - 2.6.24.5. On my Link1 account I have a 2.6.18 kernel. This means I absolutely, positively cannot use the slackware headers. Why? Because, quite simply, you never use a set of kernel headers newer than your kernel. It can result in serious breakage. Trust me, I have learned the hard way.

In the D package we also have make 3.81. Go ahead and snag it. We will use the slackware make only as long as we must before building our own.

[edit]
== A brief interlude with slackware's make ==

Installpkg it as well, please.

[edit]
=== Building the linux kernel headers ===

cd into linux 2.6.18 and make mrproper again.

It should complete without a problem.

Next, we do this:

make headers_check

We get yet another error message! We need a command called unifdef.


[edit]
==== Yet further recursed troubleshooting: unifdef ====

Unifdef was released eight years ago, and was not well maintained. Freshmeat has it, but the sad part is that both the links are http which we can not use yet. There is a son of unifdef (sunifdef) but we cannot build it. We would need perl, which we do not have. However, while slackware never included it Fred Emmott, the maintainer of a slackware build for x86_64, did include it in his slamd64 11.0 release. You can browse to this site and with a little observation, figure out how to get the source for unifdef. As you can see, he even has a newer version than the one on Freshmeat. Most of the changes, so far as I'm aware, were made so unifdef was compatible with newer ANSI C standards and thus newer versions of gcc.

./configure

This time, the command works. It successfully configures. Now, cross your fingers.

Enter this:

make

If your VPSLink1 account is like mine it completes very quickly indeed. We always want to test what we have just built, so this is what we enter:

make check

It fails all the checks! All three simple tests! However, we are missing a program called "diff". Diff is considered very important in lots of things that require comparing one program to another. However quickly you want to go "okay, lets get diff!" there is yet another problem. Diff requires the sanitized kernel headers we are trying to build in order to compile. We cannot get those without unifdef. We need diff to test unifdef, but we have a compiled unifdef program. We just cannot run the test suite.

Here is what we do now, since we're backed into a very small corner. Unless we want to get more pre-compiled packages (and we do not) we charge blindly forward and pray unifdef works.

Generally, we want to avoid this.


I'm not going to show you how I figure out where a program installs. If you really want to understand the internals of make, it has manual pages (man make) you can read.

make install DESTDIR=/tmp/make


Why did I just install our new program somewhere else, you ask. Rest easy, young padawan - We are about to make our own very first package! However, in order to keep with slackware conventions, there are a few other things we need to do.


Using less, we can read files. (We used to use a program called more, but then less was written, and it had more features then more, and it worked better, and thus we coined the phrase "less is more"! Never let it be said programmers do not have a sense of humor. It is just obscure, is all.)

We see the following:

INSTALL is a generic file. It says so on the first few lines, so we ignore it.

README is specific to unifdef, so, since slackware (as you can see with an ls) maintains a /usr/doc/programname-version/documentation type file tree, we will build one for unifdef.

ginstall /tmp/unifdef/usr/doc/unifdef-1.0+20030701 -d

Using ginstall is simpler then having to use mkdir twice. It is a nifty trick when you need to create a full directory tree that does not exist already.

But first, what else do we have? COPYING turns out to be the program license. We want to keep that around just on general principles. (Always respect the work of others, ok?)

cp {COPYING,INSTALL} /tmp/unifdef/usr/doc/unifdef-1.0+20030701

This copies both of them into the doc tree.

Now, because sometimes packages do not have sane ownership, from the directory we compiled unifdef in we are going to:

ls -l

You see how some of them say root root and some say 1000 1000? That is because some files are owned by user and group root, and some are owned by user and group 1000 which do not exist on my system. So we know we will need to change ownership of our install tree, just to be safe.

cd /tmp/unifdef
chown root.root * -R
cd usr/local
mv * ..
cd ..
rm -Rf local
cd man
ls man1

Now, we move into our directory structure that we will turn into a package. We recursively change ownership for everything. It's all roots now, period.

Then, we take a look around. To build a system package that we might choose to distribute, we do not want anything in /usr/local, so move it all into /usr. Then we remove the now empty local directory. Next we go into man, and take a look at the man1 directory. We see one lonely section 1 manual page. So we do this:

gzip man1/*
cd ..
cd bin

In the bin directory, we have one lonely file. If we file the file, we find that:

root@darkstar:/tmp/unifdef/usr/bin# file unifdef
unifdef: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
root@darkstar:/tmp/unifdef/usr/bin# 

It's not stripped. Well, unstripped, it is 34310 bytes long. We are not planning on running it through gdb anyway and debugging it, so we do this:

strip --strip-unneeded unifdef

It is now 12196 bytes long. As you can imagine, on larger binaries this can make a big difference -- especially on systems like my VPSLink1, with limited space.

Now, we do this:

cd /tmp/unifdef
mkdir install
cd install
cat - >> slack-desc << EOF
# HOW TO EDIT THIS FILE:
# The "handy ruler" below makes it easier to edit a package description.  Line
# up the first '|' above the ':' following the base package name, and the '|'
# on the right side marks the last column you can put a character in.  You must
# make exactly 11 lines for the formatting to be correct.  It's also
# customary to leave one space after the ':'.

       |-----handy-ruler------------------------------------------------------|
unifdef: unifdef is a handy program for removing #ifdef and such from a 
unifdef: .c source file.
unifdef: 
unifdef: 
unifdef: 
unifdef:
unifdef:
unifdef:
unifdef:
unifdef:
unifdef:
EOF

We have just created the description that the slackware package will use for our unifdef program. Then, we do this:

cd ..
ls -lR | less

We take one last look for any flawed permissions. We see that anyone can use or read these files, but nobody except root (not even someone else in the root group) can edit or change these files. That is what we want. The same goes for the directories, and all directories are +x for all users. If it was only +x for user and group root, then nobody else could use anything in them and they might break the permissions on everything in /usr for everyone but root! In fact, if we ls -l in /, we will see the directories there all have the same permissions except tmp - which is good and proper. So, now we are ready for the big step:

makepkg unifdef-1.0-20030701-i686-1.tgz

It will ask us if we want to sanitize permissions, but we did that by hand, so tell it no. Since we did not specify march as 386 or 486, we know it compiled it for i686 -- or we have reasonable belief that it did so. The trailing -1 means it is our first attempt at the package. Now, we installpkg it, see our carefully created description, and voila!

To test it, run unifdef --version. It will give its usage parameters. As we can see, it does not support providing its version. That is fine. We know it runs. Now, for the test:

[edit]

Finally, actually making the sanitized kernel headers!

cd into your linux kernel source directory again, and make headers_check.

*gasp*

It is working!

It is actually working!

Ah, the excitement of a risk taken, and a chicken and egg problem solved.

Now we have to install the headers. The first question is, what architecture of headers do we need? Well, we are using plain slackware, which means 32bit, and we are on an Intel CPU, so we know it's i386. So we do this:

make ARCH=i386 INSTALL_HDR_PATH=/tmp/headers headers_install

Voila! We now have sanitized headers waiting to be packaged! Whoohoo! See? Persistence and troubleshooting can fix things.

cd into /tmp/headers, and we find we have our include directory right there. We need it to be in usr/include so do this:

mkdir usr && mv include usr

ls -lR | less and check all the permissions. Mine look good to me. Now we are ready to mkdir install and make a slack-desc file in it. I will let you do it on your own. The previous snipped has instructions for using Pat's handy dandy ruler. Put whatever you want in the description, so long as it tells you what you need to know. Personally, I like to put little comments of triumph when I've successfully done something I had trouble with. This time, mine would read something like this:

These are the 2.6.18 linux kernel headers. They are required to 
compiled many things, and through much hardship and endurance,
I have built my own sanitized headers! -jmd


Then, of course, makepkg linux-headers-2.6.18-i386-1.tgz and installpkg it! (P.S. If you put personal triumphs in your descriptions, never let anyone examine /var/log/packages or run pkgtool - Inevitably, it will be someone arrogant!)

Now back before all of this happened we were trying to build make!

But you will have to go on to Part 2 for that!

Retrieved from "http://wiki.vpslink.com/Custom_Linux_Environment_-_Part_01_-_Compilers"
Recent Changes | RSS RSS Feed