DSL Tips and Tricks :: declobber  - a tool for extension rebuilding



Wouldn't make any difference other than for a botched extension containing multiple entries having the same filename and path (possible with tar), but uniq -u should be uniq in the pipe feeding dir_filter:

Code Sample

#!/bin/bash

# declobber v0.2.1 by  wdef  Feb 10 2007
# Removes clobber dirs from .dsl extensions

# Changes v0.2.1
# uniq -u -> uniq in pipe feeding dir_filter

# Changes  v0.2
# 1. If used on livecd legacy boot or equivalent, can differentiate between isolated empty clobber dir
# that also occurs on base system and one that is non-base (ie added by user or other extension).
# This is noted in the results for information.  However, isolated empty dirs are infrequent in
# extensions.
# 2. Help and message related to (1) added.
# 3. Added checks for work dir
# 4. Minor changes to output.


#==========================// USER SETTING//===============================

# Put DIR on a hard drive linux partition if insufficient room in /ramdisk/home/dsl
DIR=/home/dsl

#==========================// FNS //========================================

help(){
cat <<"EOF1"
declobber v0.2.1 - removes clobber directories from damnsmalllinux extensions
by wdef
Usage: declobber <ext1.dsl ext2.dsl ..>
Run as root to preserve all permissions.
Creates a work dir in $DIR
Declobbered extension is output in $DIR
Default DIR=/home/dsl
EOF1
exit
}

check_work_dir(){
# Check work directory looks ok
if [ -z "$DIR" ]; then "${RED}User setting work dir is unset.${NORMAL}"; exit 1; fi
case ${DIR} in
/mnt/*) D=$(echo ${DIR} | awk -F/ '{print "/" $2 "/" $3 }' )
FS=$(grep ${D} /proc/mounts | awk '{print $3}')
if [ -z "$FS" ]; then echo "${RED}$D not mounted.${NORMAL}"; exit 1; fi
case $FS in
ext*|*eiser*);;
*fat*|*dos*) echo "${RED}$DIR on $FS can't preserve perms - use linux filesystem.${NORMAL}"; exit 1;;
ntfs) echo "${RED}Writing to ntfs is not a good idea ${NORMAL}"; exit 1;;
esac;;
/*|/ramdisk/*);;
*) echo "${RED}Err .. where exactly _is_ this $DIR anyway?"${NORMAL}; exit 1;;
esac
if [ ! -d "${DIR}" ] || [ ! -w "${DIR}" ]; then
echo "${RED}$DIR not a dir or not writeable.${NORMAL}"; exit 1
fi
}

not_legacy_live_msg(){
cat <<"EOF2"

This is not a legacy livecd boot.
If your extension contains isolated empty directories that are also found
on the system, declobber cannot be certain if these are clobberring base
system directories or directories added by user/other extensions, or both.
I will note any such dirs in the results.
EOF2
}


checkon_system(){
# Remove empty dirs if already on system
# Make note in results of empty dirs that appear to clobber non-base dirs (eg added by other extensions)
# or if declobber can't tell.

if $LEGCD; then
if [ -e "/${LM}" ]; then
if [ ! -e "/KNOPPIX/${LM}" ]; then
echo ${LM} >>${clobber_list}
echo "Removing empty dir /${LM} - clobbers non-base added directory" >>${results}
else
# this *is* a system clobberring dir !
echo ${LM} >>${clobber_list}
fi
else
echo ${LM} # OK
fi
else
if [ -e "/${LM}" ]; then
echo "Removing empty dir /${LM} - clobbers either base or added directory" >>${results}
echo ${LM} >>${clobber_list}
else
echo ${LM} # OK
fi
fi
}



dir_filter(){
# Filter out non-empty dir lines from stdin
while read LM; do
if [ -d "${WORK}/${LM}" ]; then
if [ $(ls -1A ${WORK}/${LM}| wc -l) -eq 0 ]; then
checkon_system
else
# this *is* a self and/or system clobberring dir !
echo "${LM}" >>${clobber_list}
fi
else
# files, symlinks - ok
echo ${LM}
fi
done
}


#=========================// MAIN //=======================================


. /etc/init.d/dsl-functions # for ANSI colors


if [ "${1}" = -h ] || [ "${1}" = --help ] || [ $# -eq 0 ]; then help; fi

check_work_dir


if [ -e /etc/sysconfig/unionfs -o ! -e /KNOPPIX/bin/ash ]; then
LEGCD=false;
not_legacy_live_msg
else
LEGCD=true;
fi


if [ $EUID -ne 0 ]; then
while true; do
echo "${WHITE}You need to be root to conserve all permissions."
echo -n "Quit? (y/n)${NORMAL} "
read
case $REPLY in
y|y*|Y|Y*) exit 1;;
n|n*|N|N*) break;;
*) echo "Invalid response";;
esac
done
fi


results=/tmp/declobber_results


rm -f ${results} # preclean

CURRENTD="${PWD}"
APP=""


for APP in $*; do

echo ".........................................................................."
echo >>${results}
# If no path supplied, assume current working dir

if echo ${APP} | grep -q -v '^/'; then APP="${CURRENTD}/${APP}"; fi

if [ ! -e "${APP}" ]; then
echo "${RED}Can't find ${APP}${NORMAL}" >>${results}
continue
fi

if [ ${APP##*.} != dsl ]; then echo "${RED}${APP} is not a .dsl extension.${NORMAL}" >>${results}; continue; fi

NAME=$(basename ${APP} .dsl)
WORK=${DIR}/$NAME
LIST_IN=${WORK}/list_in
LIST_OUT=${WORK}/list_out
clobber_list=${WORK}/clobber_list

if [ -e "${WORK}" ]; then echo "${RED}${WORK} already exists.${NORMAL}" >>${results}; continue; fi

mkdir ${WORK}


echo
echo "${WHITE}Removing any nasty clobbering dirs from ${YELLOW}$NAME.dsl ..${NORMAL}"
echo
sleep 1

cd ${WORK}
touch "${clobber_list}" # zero count initialised
tar -zxpvf ${APP}
echo -n "${WHITE}Working ..${NORMAL} "

tar -ztf ${APP} | sort | uniq | tee ${LIST_IN} | dir_filter >${LIST_OUT} &
rotdash $!

NEWNAME=${NAME}_noclobber.dsl
tar -T ${LIST_OUT} --no-recursion --numeric-owner -cpf- | gzip -9 >"${NEWNAME}" &
rotdash $!
if [ ! -s "${NEWNAME}" ]; then echo "${RED}Problem making $NAME_noclobber.dsl" >>${results}; continue; fi

# Debug - check that clobbers removed = difference between in and out
DIFF=$(cat ${LIST_IN} ${LIST_OUT} | sort | uniq -u | cat - ${clobber_list} 2>/dev/null | sort | uniq -u | wc -l)
if [ $DIFF -ne 0 ]; then echo "Error. Removed dirs and difference between in/out unequal, exiting .."; exit 1; fi

echo "${GREEN}Now test your new ${YELLOW}${NEWNAME}!${NORMAL}"

CDRS="$(cat ${clobber_list} 2>/dev/null |wc -l)"
CLOBBERS=${CDRS##*[ ]}
echo "Removed ${YELLOW}$CLOBBERS${NORMAL} clobber dir(s) from ${YELLOW}$NAME.dsl${WHITE} --> ${GREEN}${NEWNAME}${NORMAL}" >>${results}
echo "See ${clobber_list}" >>${results}
cd ${CURRENTD}
done


echo
echo "=========================================================================="
echo
echo "${WHITE}DECLOBBER ${WHITE}RESULTS:${NORMAL}"
echo
cat ${results}
echo
echo "=========================================================================="
echo "${GREEN}Finished.${NORMAL}"

exit 0

WDef, very impressive work.

One thing I would like to suggest is to add getopts so that one does not need to edit the script to specify output directory for the results.

Here is my suggestion:

Code Sample

DIR=/home/dsl


Be replaced with:
Code Sample

DIR=/home/dsl
while getopts ":o:" opt; do
 case $opt in
 o)
   DIR=$OPTARG
   shift $(($OPTIND - ))
   ;;
 \?) echo 'Usage: declobber.sg [ -o directory ] ex1.dsl ext2.dsl ...'
      exit 1
esac
done


This way the script can be used as

declobber.sh -o /tmp myext.dsl

or even

declobber.sh -o/tmp myext.sh

while still maintaining compatibility without the -o option.

Thx Robert  - good suggestion, easier configurability!  Will add and repost.
I tried this script on gnucash.dsl, and mydsl-loaded the resulting file (gnucash_noclobber.dsl).  Next I attempted to run the exec file cd-ing to the created  /gnucash/usr/bin folder  and typing "sudo ./gnucash" into xterm.  Loading the app failed with the following error:

dsl@box:/ramdisk/home/dsl/gnucash/usr/bin$ sudo ./gnucash                       ERROR: Could not find slib/require.scm in  ("/usr/share/guile" "/usr/share/gnucash/guile-modules" "/usr/share/gnucash/scm" "" "/usr/share/guile/site" "/usr/share/guile/1.6" "/usr/share/guile" ".")

If I mydsl-load the un-declobbered version, the above required files get loaded, and now the app will run.  Maybe  I'm doing this incorrectly.

EDIT:  In comparing the created /usr/share/guile/1.6 files with the KNOPPIX/usr/share/guile/1.6 files, "slibcat" file is missing.

You're running the program "installed" under your home directory rather than under /usr. If the program contains libs or other files that it expects to find in /usr, the program won't work.
Apparently the new package got "/home/dsl/gnucash" included.  From first glance, it looks like the $LIST_OUT is incorrectly including the work directory in the filenames....maybe not, but the work directory is getting in there somehow.

Next Page...
original here.