Poor dd PerformanceForum: Laptops Topic: Poor dd Performance started by: gorn Posted by gorn on Sep. 11 2005,08:06
Guys, Need a little help with this one. Have a Dell D600 laptop with 2 40GB disk drives (1 internal, 1 in modular bay). I boot DSL via a USB stick and then use dd to copy the entire primary drive to the modular drive. Why? I travel a lot and cannot afford the downtime to reinstall / configure anything. Problem is that performance of the copy via dd is unacceptably poor. It now takes around 10 hours to copy hda to hdc. I have experimented with changing block sizes and also booting DSL to RAM, but no notable improvement. Here is a little further information. Maybe someone has had something similar to this. 15 years ago I was a UNIX admin, but in the meantime I have forgotten a hell of a lot. root@0[cdrom]# uname -a Linux box 2.4.26 #1 SMP Sa Apr 17 19:33:42 CEST 2004 i686 unknown root@0[cdrom]# lscpi 0000:00:00.0 Host bridge: Intel Corp. 82855PM Processor to I/O Controller (rev 03) 0000:00:01.0 PCI bridge: Intel Corp. 82855PM Processor to AGP Controller (rev 03) 0000:00:1d.0 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #1 (rev 01) 0000:00:1d.1 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #2 (rev 01) 0000:00:1d.2 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #3 (rev 01) 0000:00:1d.7 USB Controller: Intel Corp. 82801DB (ICH4) USB2 EHCI Controller (rev 01) 0000:00:1e.0 PCI bridge: Intel Corp. 82801BAM/CAM PCI Bridge (rev 81) 0000:00:1f.0 ISA bridge: Intel Corp. 82801DBM LPC Interface Controller (rev 01) 0000:00:1f.1 IDE interface: Intel Corp. 82801DBM (ICH4) Ultra ATA Storage Controller (rev 01) 0000:00:1f.5 Multimedia audio controller: Intel Corp. 82801DB (ICH4) AC'97 Audio Controller (rev 01) 0000:00:1f.6 Modem: Intel Corp. 82801DB (ICH4) AC'97 Modem Controller (rev 01) 0000:01:00.0 VGA compatible controller: ATI Technologies Inc Radeon R250 Lf [Radeon Mobility 9000 M9] (rev 02) 0000:02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5705M Gigabit Ethernet (rev 01) 0000:02:01.0 CardBus bridge: O2 Micro, Inc. OZ711EC1 SmartCardBus Controller (rev 20) 0000:02:01.1 CardBus bridge: O2 Micro, Inc. OZ711EC1 SmartCardBus Controller (rev 20) 0000:02:03.0 Network controller: Broadcom Corporation BCM94306 802.11g (rev 03) root@0[cdrom]# hdparm -tT /dev/hda /dev/hda: Timing buffer-cache reads: 1532 MB in 2.00 seconds = 766.00 MB/sec Timing buffered disk reads: 8 MB in 3.08 seconds = 2.60 MB/sec root@0[cdrom]# hdparm -tT /dev/hdc /dev/hdc: Timing buffer-cache reads: 1528 MB in 2.00 seconds = 764.00 MB/sec Timing buffered disk reads: 8 MB in 3.07 seconds = 2.61 MB/sec root@0[cdrom]# time dd if=/dev/hda of=/dev/null bs=512 count=1953126 1953126+0 records in 1953126+0 records out real 5m28.150s user 0m0.690s sys 4m4.080s root@0[cdrom]# time dd if=/dev/hdc of=/dev/null bs=512 count=1953126 1953126+0 records in 1953126+0 records out real 5m28.150s user 0m0.690s sys 4m4.080s Posted by NotTheMama on Sep. 11 2005,08:39
On my embedded DSL 1.4 /bin/dd is a link to /bin/busybox. Busybox is a nice program to work as multiple programs and save diskspace, but there is a downside. Not every thing works as normal. Command ps doesn't need -ef to show everything. And other commands might work differently.You could try to download gnu-utils.dsl (I think it is called this) and get the "real" dd. Maybe that works better. HTH., Remco Posted by gorn on Sep. 11 2005,11:44
Remco,Thanks for the tip. I have the same link to busybox so I'll try to get the real dd and let you know how it works out. John Posted by friedgold on Sep. 11 2005,16:17
Is dma turned on for your drives?hdparm -d /dev/hda should tell you hdparm -d 1 /dev/hda should turn it on for most drives if it isn't on already. If it is a really old PC you may have to specify the mode with -X at the same time (read the man page for more details). I think you can enable dma with the cheatcode dma as well. Posted by undertow on Sep. 11 2005,17:22
more info at wiki about hdparm too that could possibly speed things up for you:< http://damnsmalllinux.org/wiki/index.php/Hdparm%2C_using > Posted by cbagger01 on Sep. 12 2005,16:53
I also suggest that you make sure that DMA is turned ON for BOTH devices.I also prefer the real gnu-utils "dd" command over the busybox "dd" command. Posted by gorn on Sep. 17 2005,09:50
Groovy - looks like we nailed this one down. Thanks for all the good tips. Here's what I did:1. Installed the real dd command as part of the gnu-utils.dsl. Downloaded the .dsl and simply copied to the root directory - DSL recognized and loaded at boot (a lot easier than what I remember having to do under Solaris). Results using real dd: time dd if=/dev/hda of=/dev/null bs=512 count=1953126 1953126+0 records in 1953126+0 records out 1000000512 bytes transferred in 327.886628 seconds (3049836 bytes/sec) Original results using dd -> busybox: time dd if=/dev/hda of=/dev/null bs=512 count=1953126 1953126+0 records in 1953126+0 records out real 5m28.150s user 0m0.690s sys 4m4.080s Although this showed almost no improvement in the test, I can't be sure of the real effect because I have never benchmarked the drive to drive copy (forgot to time the command and don't feel like waiting 7-8 hours to do so right now). I did have the impression the the drive was maxed out doing the sequential read using the real dd - with the busybox dd it seemed to sometimes cycle. 2. Enable the dma paramater on hda according to directions above Results using real dd with dma activated: time dd if=/dev/hda of=/dev/null bs=512 count=1953126 1953126+0 records in 1953126+0 records out 1000000512 bytes transferred in 28.466377 seconds (35129181 bytes/sec) Performance improvement was at least a factor of 10! I'll do a real life test with dma activated for both hda and hdc, then post results. Thanks again to all for the help. |