Declobbered

Forum: The Testing Area
Topic: Declobbered
started by: roberts

Posted by roberts on June 05 2007,02:01

As was previously in the May extension thread, many extensions suffer from over-writing base system directories. Doing this is not tolerated well when mixed with unionfs. WDef wrote a very useful script to detect this and then strip out offending directories. Instead of waiting for users to complain about individual extensions, I have run the declobber script on all .dsl extensions in the repository.

I need volunteers to test these 'declobbered' extensions and post here, in this thread. As soon as I see positive feedback, I will move the declobbered extension to overwrite the original (bad) one. The url for testing the declobbered extensions will not be added to the extension webpage nor will it be added to the mydsl download tool.

Hopefully this url will be temporary and I will be able to move all these rebuilt extensions to their proper place. This will then provide a means for users to have a better experience using the latest version of DSL.

To be begin testing please use this temporary < url >

Thanks in advance.

Robert

Posted by curaga on June 05 2007,14:56

Um. Why is grub-splash there? I have a local copy, and it is the same as the declobbered one (Nothing was removed/altered).. Originally it has no directories, so maybe declobber has a bug..

Posted by mikshaw on June 05 2007,16:35

I am planning to build a new tkdvd uci package that uses the tcl/tk uci rather than its own version. This won't be done right away, though.

Posted by ^thehatsrule^ on June 05 2007,20:32

here's a manual test...

Code Sample

tar ztvf grub-splash.dsl | grep ^d
drwxr-xr-x 0/0 0 2007-03-22 07:58:50 ./lib/grub/i386-pc/

Posted by curaga on June 06 2007,06:38

That dir does not exist in DSL.. The DSL one is /usr/lib/grub/i386-pc so it doesn't overwrite a thing

edit: Oops. I had forgotten that. I left that there on purpose: if there was a manually installed grub (since DSL grub files are in /usr) some of it's files might stay, and 'cause they aren't compatible across different builds, using them might crash grub or seem success but prevent booting..

So I think it should be removed from the declobber list..

Posted by ^thehatsrule^ on June 06 2007,07:32

I don't understand what you mean...? Manually installed grub means to install from source?

Posted by curaga on June 06 2007,14:45

Or through apt-get. The point is that directory's existance is no harm to DSL users.
It is just a precaution if one had gotten grub from other sources

Posted by mikshaw on June 06 2007,16:04

This is exactly the kind of thing I was talking about earlier. The declobber script should not delete all empty directories in an extension. It should remove only those directories that do not already exist in DSL. Or a better solution, the mydsl-install process should not install directories if they already exist (in case that directory was created by the user and was not in the original DSL). Simply assuming all empty directories are bad is being a bully. Some applications might *require* a certain directory to exist. Users might appreciate a package builder including an empty directory as a way of saying "here's where you put add-ons for this application".

My personal belief is that simplicity has its limits of usefulness, just as complexity does. When you make decisions in order to fix a problem, it doesn't help much if your solution creates a new problem by taking a "zero-tolerance" approach (some coyotes cause harm, so kill them all?).

Then again, on the zero-tolerance side, a package builder *could* easily create a wrapper that will check for and conditionally create any necessary empty directories...

Posted by curaga on June 06 2007,16:46

Quote

It should remove only those directories that do not? already exist in DSL.

You mean that do exist ;)

Posted by ^thehatsrule^ on June 06 2007,18:01

But I don't see the need for that directory to be included in this case? It will be created with 0:0 permissions if it doesn't already exist.
The problem arises if someone creates uses a dsl-grub.unc or whatever.

Posted by mikshaw on June 06 2007,21:40

Quote

You mean that do exist ;)

Yes, exactly. It's opposite day.

Quote

But I don't see the need for that directory to be included in this case

I don't know what the case is exactly, but I suppose it's probably easier to do the thing that occured to me at the end of my rant...creating a directory from a wrapper script. If it's a uci or unc, the "directory" could be included in the package as a symlink to /opt/something_else, which would be writable.

Posted by WDef on June 08 2007,22:08

I'll look at this issue again tomorrow when I'm awake.

However I'm pretty sure I had it right the first time. The goal of the script is to help prevent extensions breaking the system or other extensions. As I recall the policy wasn't about being a "bully"; it was about the script not being able to tell whether an empty directory could potentially clobber files in eg some other unknown extension having the same non-system directory. In that case, it's a bad coyote, and I prefer a risk management approach to letting through hanging empty directories in *any* part of the writeable filesystem, with unpredictable results when installed over other extensions.

How many extensions have non-base empty directories anyway that are actually supposed to be there and that do anything at all? One in 200? The vast majority shouldn't be there. So I erred on the side of caution.

It's not a replacement for the extension builder's brain (I wish), it's just a simple coarse filter for badly built extensions, and you should preferably check/test its results, which is why this thread is here. That's also why the file lists are left in the working directory. You can always put an empty directory back. After all, you built the extension in the first place.

But as I said, I'll give it some more thought and suggestions are welcome.

Posted by mikshaw on June 09 2007,13:47

Quote

How many extensions have non-base empty directories anyway that are actually supposed to be there and that do anything at all? One in 200? The vast majority shouldn't be there. So I erred on the side of caution.

I totally agree. I know I have seen at least one application which for some reason got angry when it was missing an empty directory. I can't remembr if it failed to launch, or if it merely spit out some nasty words. But after second thoughts, it seems more logical to stick with simplicity. Declobber fully and without prejudice. If the package builder needs an empty directory for some dumb reason, she should create it at first runtime. Also, putting more emphasis on declobbering would probably be good, such as making it a requirement in the *.dsl and *.tar.gz extension-building process. Well, maybe not a requirement, but something like "If you don't know what declobbering does, you should probably run the script".

Posted by roberts on June 09 2007,22:03

Based on the discussion in this thread from our most knowldgeable users, it would appear to be a better solution for me to post the declobbered .dsl extensions. From said discussion the number of issues would be dramatically reduced. As only those extensions that acutally needed an empty directory would fault. As it is now, many .dsl extensions, as pointed out by user, jls, crash the system

Posted by roberts on June 10 2007,17:09

The declobbered extensions have been moved into their respective directories. Hopefully this will end the issue of clobbered systems.

Thanks for the feedback/discussion.