easier way to compile stuff for DSL
Forum: DSL Tips and Tricks
Topic: easier way to compile stuff for DSL
started by: yodelaye
Posted by yodelaye on June 18 2007,19:08
Hi DSL is great but has a drawback : it s a modified debian. Some debian package won't install through apt-get because DSL has not the proper debian dependency structure. This make compile stuff for DSL a real problem. There are often missing libs you can't add because of dependencies issues.
The mydsl repositery does include .dsl to compile and there was a mention on the forum of a script from Juanito to make the compile process easier. But it did not worked for me in some cases (may be because not used the proper way) but anyway this is another option, hopefully you ll find it usefull.
looking at the debian live cd stuff, I recently discovered the debian deboostrap function to install a minimal debian system. It gave me the idea to use it to build a compile environment for DSL which can be accessed by a chroot. In that way it is possible to compile stuff for DSL directly into DSL (no need to reboot into KNOPPIX or real debian)
this is a short how to to do this :
1.install debootstrap into DSL using apt-get
- install the myDSL dpkg extension
- apt-get update
- apt-get install debootstrap
- run the debootsrap install : this will download and install a minimal debian system into the directory you choose, without changing your running DSL (choose the woody install)
2. switch to your new debian by chroot :
- once the new debian environnement is installed you are able to chroot in. this means you tell your system that the root "/" of your system is not any more the one set during boot process by DSL, but the /hda3/my_debian (assuming the target of the deboostrap install was there into the my_debian directory)
- after the chroot the system will see
/hda3/my_debian/ as /
/hda3/my_debian/bin as /bin
/hda3/my_debian/etc as /etc
and so on
- to chroot simply cd /mnt/hda3/
- sudo su
- chroot ./my_debian/
- you can check that you are in a new environment :
the result of pwd is /
Note : you are looged as root in this new system
3. prepare your new system to be able to compile apps
- to compile you need to install all the required tools. these will be installed only in your new system which is completely independant from your DSL box.
- as you now work in a real debian you can easily do it using apt-get install and all dependency issues are gone : yeah !
- apt-get install gcc flex bison make automake autoconf (and so on)
- we also need to install the some dev packages of the libs the app you want to compile needs libsdl-dev for example
4. start compiling
- all the chroot stuff has been done in a terminal window, but DSL is still running outside of this window. this means you still can launch firefox, go and grab the source of your app on sourceforge, save it into your normal home/dsl directory and use emelfm to decompress it into a directory of your new system
- or in another terminal simply run from your home directory
tar -xzvf my_app.tar.gz /mnt/hda3/my_debian/usr/src/
- now you can go back in the terminal where you are chrooted in the debian system, go inside the src directory
- cd /usr/src/my_app-src/
- and perform a classic compilation procedure
- ./configure
Note: use the prefix=/opt option to have the programm fully installed there so that you can transfer it easily later to DSL and you can easily turn it into myDSL extension or store it in your persistent opt partition
- ./make
- ./make install
- once the compilation is finished you can from DSL (simply from the file manager emelfm for example) copy all the compiled files into your standard DSL
5. et voila !
I cannot say this works all the time but I found this usefull because it avoids to reboot to make your compilation work and avoid to install a multiboot with different system on your PC which can be a difficult process especially on old hardware DSL is dedicated to
Yodelaye
Posted by WDef on June 18 2007,19:54
This looks like it's got a lot of potential.How big is the minimal woody in the chroot?
Posted by yodelaye on June 18 2007,20:26
Cannot remember how big it was just after the deboostrap and installation of the compile toolsNow I apt-get installed a bit too much useless stuff to give a good figure.
(even installed X and fluxbox in the chroot woody for fun...yes kind of weird way to have fun, I know. Didnt succeed to have it working)
I think it was 150MB to 200MB with only the necessary stuff to compile. But now the woody directory is 300MB, but including the useless X.
Posted by Juanito on June 29 2007,18:59
Out of interest I tried this with Debian Sarge (i.e. oldstable now I guess) after I deleted the /usr/share/doc & man files but before adding anything:| Code Sample |
| $ du -ch | grep total 92M total |
Posted by Juanito on June 30 2007,05:25
Hmm - I started to play with this after installing using the following:| Code Sample |
| # debootstrap --arch i386 woody /tmp/debian http://archive.debian.org/debian-archive/ |
Note that I had to install to /tmp/debian and the copy to /mnt/sdb4/debian as a direct install to hd had a bunch of errors.
The first thing I noticed was:
| Code Sample |
| # apt-get update Err http://archive.debian.org woody/main Packages Something wicked happened resolving 'archive.debian.org:http' (-3 |
I guess this means you have to re-setup everything after chroot or am I missing something?
Edit
----
Got it, you need to copy the following across:
/etc/dhcpc
/etc/resolv.conf
Posted by mikshaw on June 30 2007,15:50
Before reading this, please be aware that I'm not saying this is a bad idea, but simply that there are better alternatives, in my opinion.| Quote |
| DSL is great but has a drawback : it s a modified debian. Some debian package won't install through apt-get because DSL has not the proper debian dependency structure. This make compile stuff for DSL a real problem. There are often missing libs you can't add because of dependencies issues. |
I don't consider the inability to install debian packages a drawback, but I guess that's a matter of opinion. Compiling software, however, has little to do with your package management system, and depending on libs that are provided by such a method is actually less reliable, when compiling new software, than having no package management at all.
I used to disagree with this when others told me the same thing, because I hadn't had any trouble. Over the years, though, I saw increasing problems with mixing a binary package manager with source on the same system, and today I try to avoid package managers as much as possible.
The "missing libs you can't add" problem is an inadequacy of package management (and by the fact that DSL is increasingly incompatible with Debian). These libs are intended to be used with binary programs obtained from the same repositories as the libs, and not necessarily sufficient for compiling programs. You can compile the necessary libraries you need if binary versions are incompatible with your desired application.
Relying on another distribution, such as Debian proper, for compiling applications for DSL will probably work ok in most cases, but anytime you do this you run the risk of building programs that are incompatible with DSL if your build system does not have the same or similar versions of libs and headers that were used to build the fundamental parts of DSL. I used to build myDSL packages on an old Suse box, and most worked ok in DSL. However, newer distributions have newer libs and headers (and kernels), many of which create binaries that don't work in DSL without having to include redundant libraries. Because of this, it's my belief that using DSL itself to compile programs intended for DSL is usually going to be more reliable than using Debian.
Posted by WDef on June 30 2007,17:52
I don't know, having a full woody development environment available in this or the standard way for compiling from source might come in handy.There are some things that don't seem to compile on dsl no matter what you do .. determining what's missing is not always easy unless ./configure screams it at you or the compile bombs out at something obvious. It can be very trying.
I recall for eg that gtk-gnutella compiled on woody, but not on dsl.
Which version of SuSE were you using to compile? I seem to recall that Slackware10 had a 2.4.26 kernel for a while.
The critical thing is the glibc version. If that's newer than dsl's, then there are problems on dsl with the compiled binaries.
Posted by clivesay on July 01 2007,03:41
I'm interested to see what RS has up his sleeve for 4.0. He's hinted that he's ready to move on. He's good at surprising people with innovative ideas so I'll try to be patient and see what he has up his sleeve. Maybe whatever it is, can help with the burden of compiling apps against DSL. If I could get some more up to date apps running in DSL it would sure tweak my interests again.
Posted by Juanito on July 01 2007,05:18
One issue I came upon whilst messing around with the debootstrap woody install is that linux-kernel-headers does not exist in woody - there are kernel-headers packages but the most up to date is 2.4.20.The Debian site does not list any dependencies for the linux-kernel-headers package in oldstable, but when I try to install it with apt-get, it prompts to update libc6 and a couple of other packages - unless I'm missing something, this sort of negates the idea of the debootstrap in the first place?
Posted by WDef on July 01 2007,10:16
| Quote |
| If I could get some more up to date apps running in DSL it would sure tweak my interests again. |
Good to see you're still out there Clivesay!
What apps are you particularly desirous of obtaining on dsl, so to speak?
Posted by yodelaye on July 01 2007,13:34
Hi| Quote (mikshaw @ June 30 2007,11:50) |
| there are better alternatives, in my opinion. |
Well, I should have added a clear warning with my 1st post. This was clearly no "how to" or tutorial, but just an idea which needs to be evaluated by more linux experienced users.
I also think that compiling for DSL inside a pure debian is not a clean method as it will for sure not ensure full compatibility.
DSL is based on Knoppix, and I think Knoppix comes with a complete build environment. May be it could be possible to use knoppix as the chroot instead of debian ? (the knoppix release used as a base for DSL should share the same libs, headers,...) ?
| Quote (Juanito @ July 01 2007,01:18) |
| linux-kernel-headers does not exist in woody - there are kernel-headers packages but the most up to date is 2.4.20. |
don't know why these are not provided on < http://archive.debian.org/ >
may be this < mirror > can be used instead
Posted by mikshaw on July 01 2007,14:36
Yeah, don't get me wrong; I think it's a good idea, which was the reason I added the disclaimer to the top of the post. My only real point was that it seemed as though you were suggesting that Debian was going to solve any compiling problems you have in DSL. I thought it was important to point out that "Debian" does not necessarily mean "compatible with DSL", particularly for future reference when there will likely be an even larger rift between the two distros, and that there are risks involved with mixing package management with source-based installation.I don't think that using even the Knoppix version on which DSL was based would be any better than using Debian (and it would be much larger). DSL has changed from within since that time, so you're likely to see the same issues as you would by using an arbitrary version of Debian.
The more binaries built on other distros that end up in the myDSL repository, the greater the risk of some things not working properly. I'm guilty of doing that myself, as I said, but have been trying to correct that with each update of an existing extension.
The gcc1-with-libs package is a good starting point, but it's my opinion that an increase in available made-on-DSL-for-DSL development files would ultimately be a much more reliable way to go than using Debian or deb2dsl development packages.
WDef: It was suse9.0, which has kernel 2.4.21 and (i think) gcc 3.2. I'm not sure about the glib version, but it's all older than what DSL has. Slackware 10.1 uses kernel 2.4.29. I don't know what 10.0 had. In any case, my other distros are all slowly becoming useless as DSL compilers as they and DSL change/upgrade.
Posted by WDef on July 01 2007,22:14
@Mikshaw: thanks.@Juanito: Are you confusing "linux-kernel-headers" with the actual kernel sources? You wouldn't be the first!
Warning: sermon approaching :=)
linux-kernel-headers are not the kernel headers (at least not necessarily) of the running kernel. Instead, they are the contents of /usr/include/linux which are the headers from the kernel sources against which the system's glibc was compiled. (Back to that central glibc again). So these are the headers that match the library object files.
This confusing terminology arose because, in the old days, these headers were all the same thing. glibc was compiled against the kernel that the linux pioneers were running. So a (bad) custom developed of just symlinking /usr/include/{linux,asm} to /usr/src/linux/include/{linux,asm}. Why not, it's just duplicating the same stuff, right? In fact Klaus Knopper made a "linux-kernel-headers" package that are just those symlinks I think. To which I posted on the Knoppix mail list asking for clarification about that, but never got a response.
But once you change the running kernel without recompiling glibc (which is the usual situation), then these headers are different. So the /usr/include/{linux,asm} headers have to be kept the same as those matching glibc and shouldn't change unless you change/rebuild glibc. glibc can only use the functions and calls defined by the headers it was built against. Therefore other progs on the system should use these same headers when compiling.
Which is why Linus Torvalds says we should never compile new kernels in /usr/src/linux, or if we do, we should get rid of those symlinks and make sure the contents of /usr/include/{linux,asm} do match the running glibc.
So Woody's "linux-kernel-headers" have to match its glibc, not the kernel-du-jour. At least that's my (quite probably imperfect) understanding.
This is also why the maker of the linux-kernel-headers extension in the repo says these are kernel independent or something in the info file, which is a bit of an approximation.
</end sermon>
RE: gcc1-with-libs.dsl - I know there is at least one header in there that mismatches the lib version on the system - I think from memory it's libpng2. Needs fixing.
:0
Posted by mikshaw on July 02 2007,00:25
I'm going to guess that gcc1-with-libs.dsl was created from a debian package, which could mean that there could be incompatible files within it. I'm not sure what a solution would be apart from maybe using gcc1 to create a new gcc package, and use that new package to create all subsequent libraries
Posted by Juanito on July 02 2007,03:30
| Quote |
| @Juanito: Are you confusing "linux-kernel-headers" with the actual kernel sources? You wouldn't be the first! |
No, I don't believe so - if only from the size of the files
I wasn't aware of that whole glibc thing however, which begs the question of what the "proper way" to do this is. How do you make sure /usr/include matches the running glibc?
BTW, I didn't see a glibc deb when the debootstrap woody set up (is there a way to search the packages in Woody) - but I do see many glibc binaries on the gnu site?
Posted by WDef on July 02 2007,13:39
By glibc is meant the GNU C library, or Standard C library, which is what we call libc6This why Sarge deb packages often want you to upgrade libc6 when you try to install then on dsl - a newer libc6 is a necessary dependency. It is fundamentally what defines what userspace prog will run on which system (other libs notwithstanding).
Unfortunately dsl's libc6 (from the Knoppix3.4 parent disk) is not exactly Woody's, but it's very close - close enough not to create difficulties running Woody binaries. But as we discover apt-get on Woody repos doesn't always agree, which is when I use --ignore-depends on dpkg. Knoppix sometimes customize and compile their own versions of things that are slightly different to the debian base.
| Quote |
| How do you make sure /usr/include matches the running glibc? |
I guess by having glibc and linux-kernel-headers from the same distro version, without fiddling with either. Or by recompiling glibc and putting the appropriate kernel headers in /usr/include/linux etc
| Quote |
| I'm going to guess that gcc1-with-libs.dsl was created from a debian package |
I imagine that a number of debian dev packages were added to gcc to make this.
Didn't Knoppix used to have their own repos with all their own packages? If so then the matching headers exist somewhere.
A bit of C that attempts to test compile against all the headers and libs (the way that some configure scripts do when they are testing if your headers and libs match) could systematically test all the libs and headers in gcc1-with-libs.dsl
I think I only recall coming across the one mismatch though, or it might be two - zlib ought to be checked as well.
One workaround would be to ask Robert to consider recompiling and re-installing the conflicting lib(s) but keeping the matching headers, so we have a guaranteed match - perhaps even put them into dsl. The mismatching headers could just be deleted from gcc-with-libs.dsl. Header files are often very small anyway.
A total solution would be to recompile all of dsl's libs from scratch automatically using something like the T2 build system scripts and then keep all the headers somewhere for download.
I've had some dealings with one of the authors of the T2 build system - if John and Robert want I'm sure they might consider helping set this up for dsl with Robert. It would give Robert total control over all elements in dsl and provide freedom from whatever idiosyncratic version decisions Knoppix had made.
Posted by roberts on July 02 2007,15:03
| Quote |
| A total solution would be to recompile all of dsl's libs from scratch automatically using something like the T2 build system scripts and then keep all the headers somewhere for download. I've had some dealings with one of the authors of the T2 build system - if John and Robert want I'm sure they might consider helping set this up for dsl with Robert. It would give Robert total control over all elements in dsl and provide freedom from whatever idiosyncratic version decisions Knoppix had made. |
Since DSL 4.0 has a new kernel, I would consider this for DSL v4.0. Let's see how well v4.0 is received and then if the effort is worth the result. I am hoping it will be so. I appreciate the suggestion and offer from WDef.
Posted by Juanito on July 02 2007,16:56
| Quote |
| Unfortunately dsl's libc6 (from the Knoppix3.4 parent disk) is not exactly Woody's, but it's very close - close enough not to create difficulties running Woody binaries |
This sounds like the best thing might be to use the headers from the Knoppix iso with the debootstrap woody, no? At one point I tried to achieve this using symlinks to the unpacked iso but never succeed - probably because I didn't totally know what I was doing.
| Quote |
| Since DSL 4.0 has a new kernel, I would consider this for DSL v4.0 |
This and a downloadable build environment sounds like it would be a "very good thing"
Posted by roberts on July 02 2007,17:20
This topic was visited once before by fourm member Clacker.His suggestion was to use DSL and the KNOPPIX 3.4 CD with unionfs.
This topic thread can be found < here. >
Posted by Juanito on July 02 2007,17:44
Ah-ha - very interesting thread, don't know how I missed that the first time around. Since I am more hardware orientated it's going to take me a while to get my head around the whole unionfs thing though...
Posted by WDef on July 02 2007,20:53
| Quote |
| use the headers from the Knoppix iso |
The problem (to put it another way) is I don't think knoppix3.4 contains a full development environment with all the library headers, and at least some of the core Knoppix3.4 libaries are clearly not exactly Woody versions which is one reason we can't compile some things against woody headers. Even (in one case I can think of) if I force-install woody's libpng2 over dsl's and add libpng2-dev, yes, binaries do compile - but they don't necessarily run on dsl without woody's libpng2 installed because then they are looking for the wrong library version!
I don't think from memory knoppix3.4 has the full linux-kernel-headers either, since Mr Knopper says just to do the bad symlink thing. So making a union of Knoppix3.4 and dsl with unionfs might not achieve anything.
What we need are the correct headers for all the libs. As I said I think there are about two headers in gcc1-with-libs.dsl that I've found that do not match dsl libs. There might be more. That's why I was asking what happened to knoppix's own repos - for the *-dev packages (headers).
I don't know if the linux-kernel-headers extension in the repo exactly matches the kernel headers dsl's libc6 was compiled against. Who knows. Again, I asked on a knoppix developers mail list what kernel was installed when glibc was compiled on knoppix3.4 (assuming it was) and got no response. They probably can't remember, it's a pity there is not better documentation.
The only way to fully control these things is to use a more or less unchanged distro as the base so that the key matching headers are available in the repos (in terms of core libs, leaving aside TinyX etc), or compile all the core libs from scratch so the headers are available. (I have wondered what would happen if dsl's non-woody core libs were all replaced with conforming Woody libs.)
Posted by ^thehatsrule^ on July 03 2007,00:28
Note: afaik the kernelsource.dsl includes all the kernel headers from knoppix 3.4 (i386)... yes, the name is misleading as well... should it require changing?However, I have always compiled libraries in DSL and not on the Knoppix CD, so I cannot comment on that.
Posted by WDef on July 03 2007,15:38
hats - kernelsource.dsl is only the contents of /usr/src from knoppix3.4, which does not contain the kernel sources. It's not terribly useful.linux-kernel-headers.dsl is the /usr/include{asm,linux} stuff I was talking about. But they are from linux-kernel-headers (2.5.999-test7-bk-17) unstable Sun, 6 Jun 2004 and so I'm not sure these are from Woody?
The other place to get these headers (sometimes called "protected headers" I suppose since they are kept separate from the running kernel headers) is from libc6-dev. which makes sense since it's libc6 these have to match, not the running kernel.
The poster of lib6c-dev.dsl has made linux-kernel-headers.dsl a prerequisite and factored these out, so (s)he seems to be sure these will match dsl's libc6. I suspect you need to clobber dsl's libc6 with libc6.dsl in order to make this work for compiling.
Posted by Juanito on July 08 2007,13:23
There are bunch of "appropriate sounding" kernel header debs on the site Yodelaye mentioned a few posts back in this thread, eg: kernel-headers-2.4.26-2_2.4.26-1woody.1_i386.deb.In fact I was wondering if the best thing to do to create a compile environment that was as close to/appropriate for DSL as possible, would be to compile the compiler. To this end, I installed woody debs to be be able to compile gcc, binutils and make from binaries and have been using these to build a compile environment from binaries matching the version numbers of the woody counterparts (m4, bison, flex, gettext, texinfo, libxml2, etc). Since libc6 in DSL is somewhere between woody and oldstable, I used the oldstable libc6-dev and kernelsource.dsl for headers.
Would it make sense to try and compile glibc in order to get headers for DSL's version of libc6 (assuming I can find the matching version)? Would it make sense to use the debs at the top of this post for kernel headers or would it be better to use headers from the 2.4.26 kernel source?
So far the "compiled on DSL" build environment seems to be working
Posted by mikshaw on July 08 2007,14:00
| Quote |
| I was wondering if the best thing to do to create a compile environment that was as close to/appropriate for DSL as possible, would be to compile the compiler. |
However, I don't see the point in putting any focus at all on Debian packages when building a compiling system. The two have little to do with each other, in my opinion. If it were me building a set of compiling tools, I'd probably rebuild DSL's core applications and libraries too, convert DSL into its own unique distro, and screw the package management =o)
Posted by Juanito on July 08 2007,14:29
| Quote |
| However, I don't see the point in putting any focus at all on Debian packages when building a compiling system |
What I should have said is that where a particular application/library does not exist in DSL, I used a binary closest in version number to the woody application/library in order to have something from the same era and thus likely to work with DSL/compile applications that will work with DSL
Posted by mikshaw on July 09 2007,14:38
One thing I'd forgotten to mention is a real-life example of what results from assuming that files extracted from Debian packages are good enough to pass as compatible with DSL.The ImageMagick myDSL extension has a consistent minor flaw in the gui that will present the user with a warning message if the file selector is cancelled. This happens every time i do this in a vanilla DSL. However, the bug does not show when the application is first compiled, while gcc1-with-libs is still installed. In other words, gcc1-with-libs creates applications that are not entirely compatible with vanilla DSL.
| Quote |
| I used a binary closest in version number to the woody application/library in order to have something from the same era and thus likely to work with DSL/compile applications that will work with DSL |
Unfortunately, as time passes the older debian binaries become outdated and are left behind. The only way to continue experiencing new features and fixes in these apps is either to keep DSL current (or nearly current) with Debian package management or to compile the newer versions of applications from source, specifically targeted to DSL. I'd opt for the latter, since it would be much easier on DSL development and provide more fexibility to application builders.
Posted by WDef on July 09 2007,22:56
| Quote |
| version number to the woody application/library in order to have something from the same era |
Often that isn't necessary because newer minor versions of libraries are not supposed to break binaries that depend on earlier minor versions. Then you can compile newer versions of dependent libraries than on dsl and then compile some prog that needs these against their headers and overwrite the old libraries (or keep them seperate in a uci or a different location).
Re: throwing away Debian and compiling from scratch - that's what I was getting at when I was suggesting the T2 build system to Robert.
Posted by curaga on July 10 2007,07:48
I also think compiling a new base from scratch would be best. But doing it DSL would throw away all the Debian compatibility still left. I wouldn't mind, as I never used that, and prefer compiling from source.Robert has too mentioned Debian compatibility isn't important to him..
Posted by shankus on Oct. 03 2007,04:12
hello guys,could someone please explan to a not-very-knowledgeble-person how to "compile from source" in DSL?
just wanted to install flphoto, downloaded their source code - i don't know the next step
i have dsl in debian hard drive installation, 60 mb ram, 5 gb hard disk, compaq old laptop.
cheers
Posted by WDef on Oct. 03 2007,10:49
You might want to do some googling for some beginner's guides to this.Also, always read the README and INSTALL text files in the sources for instructions.
Building from source on dsl is much like building from source on bigger systems, with the exception that dsl lacks a lot of headers and libs that may or may not be needed for particular builds. Often these have to be added first.
In the simplest scenario:
1. Install gcc1-with-libs.dsl
2. tar -zxvf somesource.tar.gz
3. cd somesource
4. ./configure
5. make
6. make install
Not all sources have a "configure" script. If they do, it will often tell you if something is missing from the system that it needs.
In the worst case scenario, you can spend some time hunting down dependencies and headers and building or otherwise installing those first, one after the other.
It can be a bit of an art form on a minimalized system such as dsl - see the thread on building Xorg as an example.
Posted by curaga on Oct. 03 2007,18:45
Or you could wait, I think florian might make an extension out of flphoto..
Posted by Juanito on Oct. 17 2007,12:30
I after a "hiatus" on this story, I finally managed to compile glibc-2.3.2 - for some reason it needs about 800MB to compile for a 13MB tar.bz2 package...So now I have gcc, binutils, glibc, m4, etc, etc compiled on dsl to /opt/build using a compile environment built from "sarge" debs (chosen because the "sarge" libc6 is closest to that of dsl). If I understand correctly, gcc requires "kernel headers" in addition to the glibc headers in order to be able to compile - I plan to take the headers from /include in the full 2.4.26 kernel sources - does this make the most sense in dsl terms?
Posted by curaga on Oct. 17 2007,12:44
The recommended procedure for getting 2.4 kernel headers:| Quote |
| make mrproper make include/linux/version.h make symlinks mkdir /tools/include/asm cp -v include/asm/* /tools/include/asm cp -vR include/asm-generic /tools/include cp -vR include/linux /tools/include |
As you see, it doesn't just copy the whole include dir..
Posted by Juanito on Oct. 17 2007,14:18
Thanks - that seems to have done the trick (at least to compile patch-2.5.9 as a test).I'm aiming to make a build/compile uci [edit] - given that some libs (eg libc-2.3.2.so) will be duplicated with dsl, is it better to symlink to the base dsl versions or use separate libs in the extension?
Posted by curaga on Oct. 17 2007,15:11
Symlinks would save space.. But how will your extension be different from gcc1-with-libs for example, which already enables compiling on dsl?
Posted by Juanito on Oct. 17 2007,16:45
Well, it would be a uci rather than dsl/unc and it would be compiled on dsl rather than (I think) made from debs.
Posted by Juanito on Oct. 18 2007,09:24
Another compile question...I noticed that I get two sets of executables with binutils/gcc, for example:
/bin/ld
/i686-pc-linux-gnu/bin/ld
The file sizes appear to be the same - are they the same, or is one optimised for i686 rather than i386?
I also get:
/bin/gcc
/bin/i686-pc-linux-gnu-gcc
again, are they the same?
Posted by curaga on Oct. 18 2007,13:25
They are the same. But for DSL you ought to compile for i486 (--host=i486-pc-linux-gnu --target=i486-pc-linux-gnu --with-arch=i486)Aren't you contradicting yourself? I mean, you compile on a Sarge debootstrap, not DSL? Or have I mixed up somewhere?
Posted by Juanito on Oct. 18 2007,13:53
| Quote |
| Aren't you contradicting yourself? I mean, you compile on a Sarge debootstrap, not DSL? Or have I mixed up somewhere? |
My starting point was to load a bunch of sarge debs (not a sarge bootstrap) in order to be able to compile gcc, glibc, etc to /opt/build and make an extension.
Next I rebooted, loaded the extension made above and used this to again compile gcc, glibc, etc to /opt/compile. Maybe this step was uneccessary but now I have a compile extension that was itself compiled on dsl
| Quote |
| They are the same. But for DSL you ought to compile for i486 (--host=i486-pc-linux-gnu --target=i486-pc-linux-gnu --with-arch=i486) |
If they are the same, which version should be deleted?
Would --host=i686-pc-linux-gnu --target=i486-pc-linux-gnu --with-arch=i486 make more sense? Is anybody going to compile with a 486?
Posted by curaga on Oct. 18 2007,14:54
They are hard links to each other. So delete both or just overwrite.Your tactic was fine, why didn't you mention that before?
--host=i686..blah would make a gcc that only runs on 686. When using --host=i486--blah it will still know you are compiling on a 686 but compile for a 486..
Posted by curaga on Oct. 18 2007,14:55
Oh, and I believe people might just compile on a 486 I compiled some on my i586 too..
Posted by Juanito on Oct. 19 2007,15:56
Hmm - it's not so obvious that the hardlinks can be deleted:| Code Sample |
| $ ../glibc-2.3.2/configure --prefix=/opt/build --enable-shared --enable-add-ons --enable-kernel=2.4.26 --disable-static --host=i486-pc-linux-gnu --target=i486-pc-linux-gnu --with-arch=i486 checking for i486-pc-linux-gnu-gcc... no checking for i486-pc-linux-gnu-cc... no checking for gnumake... no checking for gmake... no checking for make... make checking version of make... 3.80, ok configure: error: *** These critical programs are missing or too old: gcc |
Unless I make a symlink gcc -> i486-pc-linux-gnu-gcc? (rather than leaving both gcc and i486-pc-linux-gnu-gcc)
Posted by curaga on Oct. 19 2007,15:58
No need to delete them, though Hard links take less space than symlinks..
Posted by Juanito on Oct. 19 2007,17:02
| Quote |
| Hard links take less space than symlinks.. |
-really? They show up as the same size (gcc = i486-pc-linux-gnu-gcc) in emelFM...
Posted by ^thehatsrule^ on Oct. 19 2007,20:00
That's because they both are the same file on the harddisk - they have the same inode. (whereas symlinks are additional files that point to some path which takes up space)
Posted by Juanito on Oct. 20 2007,17:10
Staying with i686 for the moment - I compiled glibc, gcc, binutils, make and the 2.4.26 kernel headers into /opt/build. As a test, I rebooted and tried to compile binutils:| Code Sample |
| $ export PATH=/opt/build/bin:$PATH $ export CFLAGS=-I/opt/build/include $ export LDFLAGS=-L/opt/build/lib $ cd /usr/src/binutils-2.13 $ ./configure --prefix=/opt/compile --enable-shared --enable-threads --disable-static Configuring for a i686-pc-linux-gnu host. Created "Makefile" in /usr/src/binutils-2.13 using "mt-frag" /ramdisk/opt/build/bin/../lib/gcc-lib/i686-pc-linux-gnu/3.3.5/../../../../i686-pc-linux-gnu/bin/ld: unrecognized option '--as-needed' /ramdisk/opt/build/bin/../lib/gcc-lib/i686-pc-linux-gnu/3.3.5/../../../../i686-pc-linux-gnu/bin/ld: use the --help option for usage information collect2: ld returned 1 exit status *** The command 'gcc -o conftest -I/opt/build/include -L/opt/build/lib conftest.c' failed. *** You must set the environment variable CC to a working compiler. |
If I set cc -> gcc and/or "export CC=/opt/build/bin/gcc", I get the same error. There is also no sign of a file named conftest.c - any ideas? Does ld need to be compiled in gcc rather than binutils maybe?
Posted by mikshaw on Oct. 20 2007,21:03
| Quote |
| Well, it would be a uci rather than dsl/unc and it would be compiled on dsl rather than (I think) made from debs. |
So far it has looked like just too much work than I'm willing to spend on the project, since I have a hard enough time just getting around to doing mydsl packages that nothing else depends on =o)
Posted by Juanito on Oct. 21 2007,10:58
| Quote |
| /ramdisk/opt/build/bin/../lib/gcc-lib/i686-pc-linux-gnu/3.3.5/../../../../i686-pc-linux-gnu/bin/ld: unrecognized option '--as-needed' /ramdisk/opt/build/bin/../lib/gcc-lib/i686-pc-linux-gnu/3.3.5/../../../../i686-pc-linux-gnu/bin/ld: use the --help option for usage information collect2: ld returned 1 exit status *** The command 'gcc -o conftest -I/opt/build/include -L/opt/build/lib conftest.c' failed. *** You must set the environment variable CC to a working compiler. |
- this error turned out to be a mismatch between binutils and gcc. To fix it either:
1. Use binutils-2.15 as opposed to binutils-2.13 with gcc-3.3.5 (like debian did in sarge...)
2. Delete "--as-needed and --noas-needed" from /opt/build/lib/gcc-lib/i686-pc-linux-gnu/3.3.5/specs
The "built on dsl" compile environment now seems to compile - at least in the i686 variant. Now to move onto the compiled with compile environment built on dsl in the i486 variant.
I was thinking to include the following, the idea being to use the version that already exists on dsl or, if it doesn't exist, to use the version from sarge rather than woody (as I seem to get a lot of errors that the woody versions are too old):
autoconf-2.59
automake-1.8.5
binutils-2.15
bison-1.875
expat-1.95.2
flex-2.5.31
gcc-core-3.3.5
gcc-g++-3.3.5
glibc-2.3.2 (only headers and libs not in dsl)
glibc-linuxthreads-2.3.2
libtool-1.5.6
linux-2.4.26 headers
m4-1.4.1
make-3.80
ncurses-5.4
patch-2.5.9
pkgconfig-0.17.2
texinfo-4.7
I'll also try to look at compiling the headers for the versions of jpeg, png, tiff, zlib (and maybe) X in dsl.
Any comments on what could be added or removed?
Posted by curaga on Oct. 21 2007,12:13
Why not use gcc 3.3.6? It's a bugfix release, so it should be better to use than 3.3.5.I don't think Texinfo is needed.. Oh and use pkg-config 0.22, many packages want at least 0.20.
Posted by Juanito on Oct. 24 2007,05:44
OK, so I moved to pkg-config-0.22 and expat-1.95.8 (expat-1.95.2 would not compile itself) and compiled everything for i486 (the executables seem to be bigger than those compiled for i686). I see the next steps as:1. Compile versions of libjpeg, libpng & libtiff that match those of dsl
2. Delete the libs (but keep the headers) compiled where the versions match those of dsl
Given that I plan to compile with the eventual extension, using something like:
| Code Sample |
| $ export PATH=/opt/compile/bin:$PATH $ export CFLAGS=-I/opt/compile/include $ export LDFLAGS=-L/opt/compile/lib $ ./configure --prefix=... |
..is it OK to assume the the compilation will find the libs deleted from /opt/compile/lib in /lib & /usr/lib without symlinks (this seems to be the case)?
Edit: with all of the duplicate libs deleted, gcc will not compile shared libs - now to find which lib(s) causes this...
BTW, with the above approach "sudo make install" does not work - I need to open a terminal window as root and (re)export the path. Is there a way around this that avoids opening a 2nd window?
Posted by curaga on Oct. 24 2007,14:45
Just a note - the other extension has also gtk-1.2.10 headers..For your problem, could you try using symlinks?
Posted by Juanito on Oct. 24 2007,14:59
| Quote |
| Just a note - the other extension has also gtk-1.2.10 headers.. |
- I'm working on this, but first I need to compile XFree86-4.2.1 for the headers to be able to compile gtk (for the headers...).
tiff, zlib & libpng all compiled OK, but I need to figure out what to do with the headers/pkgconfig for the three versions of libpng on dsl. Although two of the versions put the headers in /include/libpng1 or /include/libpng12, all three of them reference the headers from /include/*.h
Posted by curaga on Oct. 24 2007,15:26
How can you tell the X files of DSL are from 4.2.1?Just curious, I always thought they were from 4.3.0, and the lib versions seem to match with both..
Posted by Juanito on Oct. 24 2007,15:32
| Quote |
| How can you tell the X files of DSL are from 4.2.1? |
- that's what it says < here >
Posted by curaga on Oct. 24 2007,16:01
Okay ;)Remember to add < ftp://ftp.xfree.org/pub....y.patch >
Posted by ^thehatsrule^ on Oct. 24 2007,16:01
If you really want to keep consistent with versions, I think XFree86 4.3.0 would be a closer version... (to the extension and tinyx servers)
Posted by Juanito on Oct. 25 2007,11:31
| Quote |
| I think XFree86 4.3.0 would be a closer version |
- I suppose so, for the XFree86 extension (4.3.0) and for the tiny-x servers (are they 4.2.1 or 4.3.0?) but the various x related libs are all 4.2.1, no?
[...and what a pain all this old stuff is to compile]
Posted by Juanito on Oct. 26 2007,10:42
I started to look at compiling XFree86-4.2.1 - I managed to patch 4.2.0 to get 4.2.1 and applied the security patch Curaga mentioned, but after that things look difficult.Does anyone have a /xc/config/cf/site.def file to use with dsl in order to compile XFree - making one based on the linux.cf does not look that obvious...
Posted by curaga on Oct. 26 2007,13:02
Here's my old Xfree 4.6.0 host.def - see if it helps you:| Quote |
| /* Begin XFree86 host.def file */ /* This setting reduces compile time a little by omitting rarely used * input devices. You can find the complete list in * config/cf/xfree86.cf *********************************************/ #define XInputDrivers kbd mouse void /* VIDEO DRIVERS **************************************************** * If you are sure you only want the drivers for one or a few video * cards, you can delete the drivers you do not want. ***************/ #define XF86CardDrivers fbdev i810 vesa vga XF86OSCardDrivers XF86ExtraCardDrivers /* These settings ensure we use our libraries ************************/ #define HasFreetype2 YES #define HasFontconfig YES #define HasExpat YES #define HasLibpng YES #define HasZlib YES /* GENERAL SETTINGS */ #define SystemManDirectory /usr/share/man #define DocDir /usr/share/doc/xfree86-4.6.0 #define BuildCyrillicFonts No /* Installation prefix. The default is to install into /usr/X11R6. If you * would like to install into /usr, uncomment these options. */ /* #define ProjectRoot /usr */ /* #define LinkGLToUsrLib NO */ /* #define LinkGLToUsrInclude NO */ /* End XFree86 host.def file */ |
Posted by Juanito on Oct. 26 2007,13:14
Thanks for that, unfotunately it gets me to the same point as the hosts.def I was trying to make. If I do:| Code Sample |
| $ make World >& world.log |
Then the last few lines of world.log look like:
| Quote |
| make[2]: Leaving directory `/KNOPPIX/usr/src/xc/config/imake' rm -f ./config/makedepend/Makefile.proto ./config/imake/imake -I./config/cf -s ./config/makedepend/Makefile.proto -f ./config/makedepend/Imakefile -DTOPDIR=../.. -DCURDIR=./config/makedepend ./config/imake/imake: No such file or directory ./config/imake/imake: Cannot exec /lib/cpp. Stop. ./config/imake/imake: Exit code 1. Stop. make[1]: *** [config/makedepend/Makefile.proto] Error 1 make[1]: Leaving directory make: *** [World] Error 2 |
The file /usr/src/xc/config/imake/imake exists and /lib/cpp -> /opt/compile/lib/cpp -> /opt/compile/bin/cpp so I'm not sure which file cannot be found...
Edit: It was a typo in the cpp symlink - now it halts complaining there's no fonts, even though I set "#define BuildFonts NO"...
Posted by Juanito on Oct. 26 2007,17:23
@curagaI guess site.def must have been modified to suit the way your host.def is set up - is it possible to paste the site.def you used? I appear to have some duplication/mismatch between the two that prevents the make completing sucessfully.
Posted by curaga on Oct. 27 2007,06:29
Sorry - I don't have that anymore..I don't think I touched any other config file than host.def though..
Posted by Juanito on Oct. 27 2007,06:58
I tried editing site.def so all it contained was:| Quote |
| /* if you want host-specific customization, this is one way to do it */ /* #ifndef SiteIConfigFiles #define SiteIConfigFiles $(IRULESRC)/host.def #define LocalConfigFiles host.def #endif */ |
- and modified curaga's host.def so part of it looked like:
| Quote |
| //* GENERAL SETTINGS */ #define SystemManDirectory /opt/compile-3.3.5/man #define DocDir /opt/compile-3.3.5/share/doc #define BuildFonts NO /* Installation prefix. The default is to install into /usr/X11R6. If you * would like to install into /usr, uncomment these options. */ #define ProjectRoot /opt/compile-3.3.5 /* #define LinkGLToUsrLib NO */ /* #define LinkGLToUsrInclude NO */ |
- in this context, I believe the "#" have to be left in, but maybe I'm wrong? In any case, "make" ignored the BuildFonts=NO and dirs=/opt/compile-3.3.5/...
Somwhere along the way, I found xfree would not compile fully without changing bison-1.875 to bison-1.875a (I found bison-1.875d, it worked) and flex-2.5.32 to flex-2.5.4 (I found flex-2.5.4a, it worked).
Once compiled, make install copied what looks like all of the headers & libs, but failed asking for /include/linux/autoconf.h - this was not made by the kernel headers commands curaga suggested a few posts back, is there a way to make it?
I copied autoconf.h from knoppix-3.4 and make install failed looking for ../lib/GL/dri/drm/xf86drm.o which I guess could be expected because I had not unpacked the full kernel sources.
I'll try moving the headers/libs from /usr/.. to /opt/compile/... and see if I can compile gtk-1.2
Posted by curaga on Oct. 27 2007,07:19
Oops. it does come from the kernel.It is created when you configure your kernel, and just creating an empty one should have sufficed..
Posted by Juanito on Oct. 27 2007,09:40
Doing this seemed to solve the problem (using the knoppix-3.4 .config):| Code Sample |
| $ cd /usr/src/linux-2.4.26 $ make mrproper $ make include/linux/version.h $ make symlinks $ make menuconfig |
This made /include/linux/autoconf.h in addition to all the other headers
Posted by Juanito on Oct. 27 2007,10:57
Hmm - compiling libfreetype6 "2.1.2-9" as per "dsl packages" gives libfreetype.so.6.3.1 whereas dsl-3.4.4 contains libfreetype.so.6.3.4, which (after some experimentation) equates to freetype-2.1.5I guess some of the libs have been updated since the packages list was made?
Also, although libexpat is not mentioned in "dsl packages", dsl-3.4.4 contains libexpat.so.1.0.0 - expat-1.95.8 gives libexpat.so.0.5.0 and expat-2.0.1 gives libexpat.so.1.5.2
Does anybody know where libexpat in dsl came from?
Posted by curaga on Oct. 27 2007,11:31
A hacked Debian one?
Posted by Juanito on Oct. 27 2007,11:50
'should've thought of that - oldstable libexpat1 (1.95.8-3) contains libexpat.so.1.0.0...
Posted by Juanito on Oct. 27 2007,14:39
It looks like the /opt/compile extension is working reasonably well to compile applications, but I seem to have a problem in compiling the kernel:| Code Sample |
| $ make mrproper $ make include/linux/version.h $ make symlinks $ make menuconfig $ make dep gcc -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -o scripts/mkdep scripts/mkdep.c scripts/mkdep.c: In function `add_path': scripts/mkdep.c:221: error: `PATH_MAX' undeclared (first use in this function) scripts/mkdep.c:221: error: (Each undeclared identifier is reported only once scripts/mkdep.c:221: error: for each function it appears in.) scripts/mkdep.c:221: warning: unused variable `resolved_path' make: *** [scripts/mkdep] Error 1 |
../include/linux/limits.h has PATH_MAX set, but ../lib/gcc-lib/i486-pc-linux-gnu/3.3.5/include/limits.h does not. Is it possible gcc is using this file instead of /include/linux/limits.h?
The only other thing I can find about this is "some toolchains don't have PATH_MAX in <limits.h>, use an additional #include <sys/params.h>", whatever than means...
Posted by curaga on Oct. 27 2007,16:22
Try adding that include to the start of scripts/mkdep.c..
Posted by Juanito on Oct. 27 2007,17:32
Adding #include <sys/param.h> to /scripts/mkdep.c does the trick - thanks.I hadn't noticed it before but, compiling the kernel does not seem to recognise export CFLAGS=-I/opt/compile/include - I had to symlink /usr/include -> /opt/compile/include to get make menuconfig to work. This probably explains (somehow) why #include <sys/param.h> is needed when compiling from /opt whereas it is not needed when using gcc1-with-libs or debs.
The best way I can think of to give the user a choice of which libpng headers to use with an eventual uci is to use symlinks ../include/libpng.h -> /somewhere/libpng.h. The user would then modify /somewhere/libpng.h to either:
../include/libpng1/libpng.h
../include/libpng12/libpng.h
../include/libpng2/libpng.h
Is there a better way to do this? What is the best location for /somewhere - /tmp, /var, /home/dsl, or?
Posted by curaga on Oct. 27 2007,18:08
/tmp, IMO.The kernel ignores CFLAGS on purpose. Suppose you got something like -march=athlon64 for normal compiling for speed. Then you need a kernel for 486. And of course forgot to unset cflags.
This happens more often than not
Posted by Juanito on Nov. 01 2007,12:01
I went with /tmp and submitted the extension - thanks for the help.