Tuxmark: 2008

Remaster Knoppix Overview

Files:
Get the bash scripts
http://pclug.info/meeting_0056/remasterKnoppixFiles.tar.gz

Credits:

Knoppix Remastering Howto
http://www.knoppix.net/wiki/Knoppix_Remastering_Howto

PCLug
http://pclug.info/meeting_0051/slide_03.html


Goals:

Have a portable personalized remaster of Knoppix

multimedia apps that I like to use
window manager other that KDE
access to virtual machines using virtualbox
boot off of a usb drive



Commentary:

The basic remastering process is what it is. I've written
some bash scripts to semi-automate the process. These
scripts were developed out of the necessity of the trial
and error process that I went through.


Setting up for remastering

The script below will copy the files that you need from
the CD to the local filesystem.

Boot your machine from a Knoppix CD, mount the partition
where you want to host your remastering. Run the following
script. Be sure to set the variables to the correct values
before execution.

#!/bin/bash
# setupForRemastering.sh
#
# boot from the Knoppix CD, open a root shell, and
#then run this script
# to setup an environment for remastering

# constants
DEVICE='/dev/hda5'
MOUNTPOINT='/media/hda5'
ROOTDIR="$MOUNTPOINT""/knx"

# find a partition to work in the
# partition should have a minimum of 3GB free space

# mount the partition
#mount -rw "$DEVICE" "$MOUNTPOINT"

# create a root directory to work in if the above
# mountpoint is not the root directory that you
# want to work in
#mkdir "$ROOTDIR"

# make two directories, one for the new master cd,
# and one for the source. also make additional
# directories under these named KNOPPIX
mkdir -p "$ROOTDIR""/master/KNOPPIX"
mkdir -p "$ROOTDIR""/source/KNOPPIX"

# copy the KNOPPIX files to your source directory
# this will take a long time. you are copying 2GB
cp -Rp /KNOPPIX/* "$ROOTDIR""/source/KNOPPIX"

# copy the 'boot' folder from your cdrom
# you need these to build the ISO later
cp -ar /cdrom/boot "$ROOTDIR""/master/"

# for Knoppix 5.0.1 you also have to copy the
# 'modules' folder
cp -ar /cdrom/KNOPPIX/modules \
"$ROOTDIR""/master/KNOPPIX/"

# copy the main HTML page for the startup page
cp /cdrom/index.html "$ROOTDIR""/master/"

# copy everything necessary except the ~700MB KNOPPIX
# file
cd /cdrom && find . -size -10000k -type f -exec \
cp -p --parents '{}' "$ROOTDIR""/master/" \;
Setup the environment for chroot

To add packages while remastering, you will need
internet access. To use the internet, we will copy
resolv.conf from the host machine to the remastering
environment. We do this right before chroot so
we capture the current nameserver of the host machine.

Then bind some mountpoints necessary for remastering
and enter the chroot environment.

The script below should be located at the root of
the remastering directory tree.

#!/bin/bash
#chroot.sh

# copy resolv.conf into the chroot folder
cp -a /etc/resolv.conf source/KNOPPIX/etc/dhcpc/

# bind temp vars
mount --bind /dev source/KNOPPIX/dev
mount --bind /proc source/KNOPPIX/proc
mount --bind /sys source/KNOPPIX/sys

# chroot into source home directory
chroot source/KNOPPIX

Modify packages

Below are the scripts that I use to modify
the packages in my remaster. The script,
aptConfigChanges.sh, is built like a stack.
Start at the bottom and move up.

The below script should be located at
ROOTDIR/source/KNOPPIX.

#!/bin/bash

#aptConfigChanges.sh

###################################################
# DO NOT REMOVE the following packages
#mutt

# DO NOT INSTALL the following packages
#nfs-common nfs-kernel-server syslinux xdm sysv-rc
#sysvinit sysvinit-utils udev
###################################################

#aptitude update
#apt-get update
###################################################

# purge 8th pass
#aptitude purge eterm qterm wterm rxvt-beta lynx \
scantv xawtv xawtv-plugins configure-xawtv pia \
nxclient unp unzip vrms vgabios locales \
util-linux-locales scribus nessus nessusd \
nessus-plugins startnessus-knoppix \
gaim gaim-data gaim-encryption gnome-icon-theme \

# install 8th pass
#aptitude install x11-common xorg xserver-xorg \
xserver-xorg-core xserver-xorg-video-all \
xserver-xorg-video-fbdev xserver-xorg-video-i810 \
xserver-xorg-video-neomagic xserver-xorg-video-savage \
ajaxterm xterm xtermcontrol xtermset whiptail wodim \
mplayer openvpn ffmpeg flac flac123 vorbisgain mplayer \
update-inetd util-linux xdelta xdialog xmms xpdf \
xpdf-common xpdf-reader xpdf-utils xsane xsane-common \
xmms xterm vim vim-common vim-gtk vim-gui-common \
vim-runtime vnc-common tar tcpd tcpdump tk8.4 tshark \
wireshark wireshark-common tzdata centericq \
centericq-common samba samba-common sane sane-utils \
scrollkeeper sendfile smbclient smbfs ssh ssl-cert \
rdesktop readline-common rsync passwd perl perl-base \
perl-modules perl-suid php4-common playmidi portmap \
ppp pppconfig pppoeconf psfontmgr python2.4 \
python2.4-minimal openbsd-inetd openssl nessus-plugins \
netbase netpbm ntpdate man-db mkisofs mknbi \
module-init-tools mount

###################################################

# purge 7th pass
#aptitude purge imaze fortune-mod fortunes-min \
aalib-bin php4-mysql autotrace libautotrace3 bb \
bochsbios dante-client dict ding euro-support \
euro-support-console gocr gsm-utils ncpfs radio \
user-de xaos

###################################################

# install 6th pass
#aptitude install linux-kernel-headers \
linux-sound-base login loop-aes-utils lsb-base

###################################################

# install 5th pass
#aptitude install icedax imagemagick initramfs-tools \
initscripts iputils-arping iputils-ping \
iputils-tracepath hal hdparm hpijs gaim gim-data \
genisoimage gimp gimp-data gpgv gpm grub grub-disk \
gs-esp gv gcc gcc-3.3-base gcc-4.1 gcc-4.1-base \
fileutils findutils fontconfig fontconfig-config \
foomatic-filters e2fslibs e2fsprogs dialog elinks \
file firefox dpkg dpkg-dev dselect debconf \
debconf-utils debian-archive-keyring debootstrap \
cups-pdf cupsys cupsys-bsd cupsys-client cupsys-common \
cron console-common console-data coreutils cpio \
cdda2wav cddb cdrecord adduser apt apt-utils \
alsa-base alsa-utils
###################################################

# remove 4th pass
#aptitude purge kbackup kbackup-multibuf \
keyboard-tr-knoppix mikmod ibod ipppd isdnactivecards \
isdnlog isdnutils isdnutils-base isdnutils-xtools \
isdnvboxserver libcapi20-2 pppdcapiplugin speex joe \
irda-common irda-tools twm zile nedit nano
###################################################

# remove 3rd pass
# remove KDE packages
#aptitude purge ark k3b kaffeine kalarm kamera kandy \
kappfinder kate katomic kbattleship kcalc kcharselect \
kcontrol kdelock-knoppix kde-menu-knoppix \
kde-services-knoppix kdeaddons-kfile-plugins \
kdeadmin-kfile-plugins kdebase kdebase-bin \
kdebase-data kdebase-kio-plugins \
kdegraphics-kfile-plugins kdelibs-data kdelibs4c2a \
kdemultimedia-kfile-plugins kdemultimedia-kio-plugins \
kdenetwork-kfile-plugins kdepasswd \
kdepim-kfile-plugins kdepim-kio-plugins kdeprint \
kdesktop kdessh kdf kdm kfax kfind kfloppy kget \
kghostview khelpcenter khexedit kicker kiconedit \
klaptopdaemon klipper kmahjongg kmail kmailcvt \
kmenuedit kmix knode knoppix-menu knotes konq-plugins \
konqueror konqueror-nsplugins konsole konsolekalendar \
kooka kpackage kpager kpdf kpersonalizer kpf kppp krdc \
kreversi krfb ksmserver ksnapshot ksokoban ksplash \
ksudoku ksync ksysguard ksysguardd ksysv kteatime \
ktimer ktip ktnef ktuberling kuickshow kuser kview \
kviewshell kwin kwlan libk3b2 libkcal2b libkcddb1 \
libkdegames1 libkdepim1a libkleopatra1 libkmime2 \
libkonq4 libkpimidentities1 libkscan1 libksieve0 \
libktnef1 libmimelib1c2a lisa networkstatus vimpart

##################################################

# remove 2nd pass
#aptitude purge synaptic wvdial modemtool-knoppix \
openoffice-de-en openoffice.org-debian-menus \
tty-openoffice icedove frozen-bubble \
frozen-bubble-data clamav clamav-base \
clamav-freshclam lftp ftpd trans-de-en brltty \
manpages-de mysql-client mysql-client-5.0 mysql-common \
amanda-client amanda-server amanda-common squid \
squid-common squidclient apache2 apache2-mpm-prefork \
apache2-utils apache2.2-common

# install 2nd pass
#aptitude install virtualbox abiword atanks vsftpd \
openssh-server openssh-client chkrootkit chntpw \
xmms-flac xmms-mp4 xmms-oggre xmms-xmmplayer ripperx \
screen jpilot

####################################################

# removed at initial remaster
#apt-get remove --purge wine kde-i18n-de kde-i18n-es \
kde-i18n-fr kde-i18n-it kde-i18n-ja kde-i18n-nl \
kde-i18n-pl kde-i18n-tr k3b-i18n sun-java5-bin \
sun-java5-demo sun-java5-jdk sun-java5-jre \
sun-java5-plugin qemu

# installed initial configuration
#apt-get install aptitude gpg gnupg

The below script should be located at
ROOTDIR/source/KNOPPIX. It is used to get
public keys for certain apt repositories.

#!/bin/bash

#getAptPublicKeys.sh

# get virtualbox public key
wget -q http://www.virtualbox.org/debian/innotek.asc \
-O- | apt-key add -

# get debian-multimedia-keyring
apt-get install debian-multimedia-keyring

# get debian-unofficial public key
wget http://ftp-master.debian-unofficial.org/other/ \
openpgp/archive-key-2008.asc -O - | apt-key add -

# get debian backports public key
wget -O - http://backports.org/debian/archive.key | \
apt-key add -

The below script should be located at
ROOTDIR/source/KNOPPIX. It displays the size
of the packages installed

#!/bin/bash

# querySizeOfPackages.sh

dpkg-query -W --showformat='${Installed-Size}\t \
${Package}\t${Description;60}\n' | sort -n

The below script should be located at
ROOTDIR/source/KNOPPIX. It removes packages
that have been orphaned

#!/bin/bash

#removedOrphanedPackages.sh

deborphan | xargs apt-get -y remove

Leave the chroot environment

The below script should be located at
ROOTDIR/source/KNOPPIX. It cleans up apt
packages and umounts /proc. After running
the below script, press CTRL+D to exit.

#!/bin/bash

#leaveChroot.sh

aptitude clean

umount /proc

After pressing CTRL+D, you will exit the chroot
environment. Then run the below script to umount
the previously bound mountpoints.

#!/bin/bash

#unchroot.sh

# release the binding of the temp vars
umount source/KNOPPIX/dev
umount source/KNOPPIX/sys
umount source/KNOPPIX/proc

mount

Other Customizations

sources.lst

# official stable
deb http://ftp.debian.org/debian/ stable main contrib
deb-src http://ftp.debian.org/debian/ stable main \
contrib

deb http://security.debian.org/ stable/updates main \
contrib non-free
deb-src http://security.debian.org/ stable/updates \
main contrib

# unofficial
#deb http://ftp.debian-unofficial.org/debian/ etch \
main contrib non-free restricted
deb http://ftp.debian-unofficial.org/debian/ stable \
main contrib non-free restricted

# supplementary repositories

# backports
deb http://www.backports.org/debian etch-backports \
main contrib non-free

# virtual box
deb http://www.virtualbox.org/debian etch non-free

# for java
deb http://ftp.us.debian.org/debian stable main \
contrib non-free

## Christian Marillat's Mult-Media Repository
## for Unstable
deb http://www.debian-multimedia.org etch main

Change default window manager

Open /etc/init.d/knoppix-autoconfig and modify
the line
[ -n "$DESKTOP" ] || DESKTOP="kde"

I changed my default window manager to icewm.


Change the boot message

Open ROOTDIR/master/boot/boot.msg and modify.
I added my name, the date, and a remaster tag.



Remastering the ISO

Below is the script I use to actually remaster the ISO.
This script encompasses the final steps of the Knoppix
Remastering Howto. I do copy the aptConfigChanges.sh
out of source/KNOPPIX and I backup the previous
knoppix.iso that I generated from a previous remastering.

I need to add some variables so this is more readable.

#!/bin/bash
#Remastering the ISO

#We've finished customizing and ready to burn!
#First do some cleanup:

# * remove any .bash_history files, tmp files, etc.
# * back out any changes you don't want burned back
# to the disc
echo ""
echo "remove .rr_moved from source/KNOPPIX"
rm -rf source/KNOPPIX/.rr_moved

echo "copy aptConfigChanges from source/KNOPPIX to ."
cp source/KNOPPIX/aptConfigChanges.sh .

echo "cp knoppix.iso to iso/knoppix.rem05.iso"
cp knoppix.iso iso/knoppix.rem05.iso


#KNOPPIX Compressed Image

echo ""
echo "make the compressed filesystem"
mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher \
"KNOPPIX www.knoppix.net" -hide-rr-moved -cache-inodes \
-no-bak -pad /knx/source/KNOPPIX | nice -9 \
/usr/bin/create_compressed_fs - 65536 > \
/knx/master/KNOPPIX/KNOPPIX

#Live CD ISO

echo "make the final cdrom image"
cd /knx/master
find -type f -not -name md5sums -not -name boot.cat \
-not -name isolinux.bin -exec md5sum '{}' \; > \
KNOPPIX/md5sums

#Now to burn the image
echo "burn the cdrom image"
mkisofs -pad -l -r -J -v -V "KNOPPIX" -no-emul-boot \
-boot-load-size 4 -boot-info-table -b \
boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat \
-hide-rr-moved -o /knx/knoppix.iso /knx/master

#(the ISO is stored in /mnt/hda1/knx/knoppix.iso)

Test the remastered iso

I usually try to boot up the remaster iso in virtualbox
to verify that my remaster is good. If it is good,
I will copy the knoppix remaster to a usb drive. If it
is not good, I'll extract a knoppix source tree from a
previous remaster that I know is good.

Both of those steps are described below.



Copy the Knoppix remaster to usb drive

Below is the script I use to copy the necessary Knoppix
files to a usb drive. Modify the path to the mountpoint
for your own use.

#!/bin/bash

#copyKnoppixMasterToUsb.sh

echo ""
echo ""
echo "ls -lah /media/usbfat"
ls -lah /media/usbfat

echo ""
echo "rm -drf /media/usbfat/knoppix"
rm -drf /media/usbfat/knoppix

echo ""
echo "cp -R master/KNOPPIX /media/usbfat"
cp -R master/KNOPPIX /media/usbfat

echo ""
echo ""
echo "ls -lah /media/usbfat/knoppix"
ls -lah /media/usbfat/knoppix

One Step Backward

If you'll remember, one of the lines in the remaster iso
script, backups up a previously remastered iso. When I
totally hose up a remaster, since I have a backup remaster
that works, I can get back to a good source. Below is the
script I use to extract a Knoppix source tree from a
compressed iso. Modify the variables for your own use.

#!/bin/bash

#getKnoppixSourceFromCompressedIso.h

COMPRESSED_ISO="/knx/iso/knoppix.rem05.iso"
MOUNTPOINT1="/media/iso"
MOUNTPOINT2="/media/image"
TEMP_ISO="/knx/tmp.iso"
KNOPPIX_SRC="/knx/source/KNOPPIX"

echo ""
echo "mount compressed iso"
mount -o loop "$COMPRESSED_ISO" "$MOUNTPOINT1"

echo ""
echo "extract compressed iso to temporary iso"
extract_compressed_fs "$MOUNTPOINT1""/KNOPPIX/KNOPPIX" \
> "$TEMP_ISO"

echo ""
echo "umount compressed iso"
umount "$MOUNTPOINT1"

echo ""
echo "mount temporary iso"
mount -o loop "$TEMP_ISO" "$MOUNTPOINT2"

echo ""
echo "remove knpppix source"
#rm -drf "$KNOPPIX_SRC""/*"
rm -drf source/KNOPPIX/*

echo ""
echo "list mounted temporary iso"
ls -lah "$MOUNTPOINT2"

echo ""
echo "copy knoppix source from mounted temporary iso \
to chroot location"
#cp -Rp "$MOUNTPOINT2""/*" "$KNOPPIX_SRC"
cp -Rp /media/image/* source/KNOPPIX/

echo ""
echo "umount temporary iso"
umount "$MOUNTPOINT2"

echo ""
echo "remove temporary iso"
rm -drf "$TEMP_ISO"

Future Modifications

Virtual Box is not working correctly
Customize icewm
Customize Knoppix Kernel
Eliminate duplicate functionality (packages)
Remaster from within a remastered knoppix

Tuxmark

Wednesday, November 12, 2008

iceweasel not finding jre

Thursday, August 7, 2008

Faster USB Transfers

Wednesday, July 23, 2008

Useful IPs

Thursday, May 8, 2008

Remaster Knoppix Overview

Friday, February 29, 2008

usb devices in virtualbox on ubuntu 7.10

click for a good time

history repeats itself