Tuning DD for server image

Jeff Boyce jboyce at meridianenv.com
Tue Mar 31 11:52:50 CDT 2009


Date: Mon, 30 Mar 2009 19:35:09 -0400 (EDT)
From: "J. Epperson" <Dell at epperson.homelinux.net>
Subject: Re: Tuning DD for server image
To: linux-poweredge at dell.com
Message-ID:
<917802f46299f74b1312ec92e3f081da.squirrel at epperson.homelinux.net>
Content-Type: text/plain;charset=iso-8859-1

On Mon, March 30, 2009 12:34, Jeff Boyce wrote:
> Greetings -
>
> I have been testing methods for making a backup image of the server for
> my small company.  I am limited to using the equipment that I have at
> hand, which consists of a single server (Dell PE 2600, RHEL3 setup as
> Samba file server with a raid5), a desktop connected to the network, and
> a portable USB hard drive.  My objective is to have a complete image of
> the entire server onto the portable USB drive (not just Samba shares)
> that can be restored onto the server with minimum additional tweaking; if
> a restoration were required.  I am preparing to complete some firmware
> upgrades soon, and would like a good image of server just in case (I have
> tape backups of data directories).  I have tried G4L and it has not been
> successful, and I am aware of Mondo, but I don't have equipment to test
> it with.  I have settled on using dd and sending it across the network to
> the portable USB hard drive mounted on the desktop.  The server has an
> old usb connection so it would take about 30 plus hours to transfer an
> image if the portable USB drive is directly connected to it.
>
> My basic usage of dd for this is  dd if=/dev/sda of=/dev/sdb bs=xxxx
> conv=noerror,sync
>
> I have tried a bs=8192 and piping it using netcat to the portable USB
> drive without success (could not see all the partitions when viewing the
> drive in GParted, nor mount anything.  Is it an inappropriate block size
> that messed up this transfer?).  I have tried not specifying a bs=xxxx
> and sent it through ssh with limited success (could see all the
> partitions, however GParted identified bad superblock errors).  I tried
> specifying a bs=512 and sending it through ssh, and this eliminated the
> superblock errors, however subsequently booting from the USB would hang
> with a kernel panic.
>
> All of these block size parameters have been suggested to me by others,
> and I am not sure how I would determine what would be the best for me in
> this situation.  I did a lot of reading on the net this weekend about
> this and could not come to any good understanding (other than it is a
> speed vs. accuracy trade off).  I am not that concerned about minor
> variations in potential speed across the network because I know that
> sending it across our network to the USB is going to be far faster than
> directly connecting the USB to the server (my estimates this weekend show
> about 6MB/sec vs. direct connection of 1MB/sec).  Can someone explain to
> me what would be a good block size parameter for me in this situation; or
> better yet, explain to me how I could understand and determine what is
> best for my situation? Thanks for any and all suggestions.

I've used g4l for several years, on 2500s through 2800s, and the only
problems I've ever had were that sometimes I've needed to try several of
the kernels to get one that supported the PERC on a given generation.  I
believe .17 through about .20 will work on a 2600, perhaps later ones as
well. It's my production imaging solution for a full stable of Poweredges.
 Whenever we go to a new RHEL/platform combination, I do a base RHEL
install on one machine, work out the driver and multipath issues and image
the others from it.

What problems did you have?  Mike Setzer, the maintainer, is _very_
responsive.  BTW, I've imaged machines that won't boot g4l, like my wife's
old Mac Powerbook, by booting a processor specific live cd (Knoppic
Powerpc in that instance), NFS mounting the filesystem I want to write to,
and doing a straight
dd if=/dev/sda|lzop > /mnt/image.lzo
But you can't readily mount the compressed disk image if that's a
requirement.


John -
I actually looked at using G4L based on your previous suggestions, and was 
optimistic about using it for this task.  Since my equipment is limited, and 
being able to test some of these things on the only server in the office is 
even more limited (weekends only), I have been running tests using my 
equipment at home at night.  So I admit that I haven't actually tried G4L on 
my office PE2600.  My home server is an old Dell Pentium 4 and my desktop is 
a new Optiplex, running Vista with Filezilla ftp server, and this is the 
setup that I was having problems using G4L.  The error I was receiving in 
the FTP log was that the connection was being dropped, about 20 percent of 
the way through the transfer.  On the next attempt, I started Wireshark to 
watch the process and the same thing happened, but also Wireshark crashed at 
that same point (first time I have seen Wireshark crash).  I was able to 
repeat this a couple of several times, then gave up and went on to try dd 
piped through ssh, which has been successful in my initial tests.  As a side 
note I don't understand why the same dd piped through netcat is giving me 
different results than dd piped through ssh.  After I am done with this task 
I will probably post this scenario and see if someone can explain to me the 
difference between them, or identify what I was doing wrong with netcat. 
Thanks.

Jeff Boyce
www.meridianenv.com



More information about the Linux-PowerEdge mailing list