UNIX Socket FAQ

A forum for questions and answers about network programming on Linux and all other Unix-like systems

You are not logged in.

  • Index
  • » General
  • » How to clone a bootable Linux machine?

#1 2008-11-13 08:41 PM

felix
Member
Registered: 2003-12-04
Posts: 171

Re: How to clone a bootable Linux machine?

Hi all DeveloperWeb users and power administrators,

As always happen, I'm here again with a new question, where I had looked before at google and some sites and couldn't found the answer, so my last chance are you guys. :)

Hope someone can help me and sorry for inconvenience of this long post....

Well, at where I work we have a old machine (512MB and HD of 20GB) running a RedHat 9 Shrike (I have minimal experience with RedHat), this machine is here before I get into the company...

This machine was acquired long years ago as a blackbox product, no one know exactly what it does, the root password, grub password, etc.

The problems are:

- The company who was the owner of this product do not exist anymore (so we have no support, no nothing).

- We had 2 box of this, one was backup (a exactly copy - who was acquired with manufacture), however recently one of then (the primary) got serious problems of HD and it do not work anymore. Now we have only one (backup) and we are using it.

And yes, this box is very important to the company. Incrediable, ahn? Yes, for me too. :)

As you should imagine my task is clone this box and reproduce it in a new one...

My first idea was get a exactly hardware and copy it (via dd), I got a exactly hardware (motherboard model, HD and memory).

My idea as use the some hardware because I do not know if it have some protections (like serial of motherboards, hd models, etc), people at my job spoke in the past they tried to clone it without success, the copy of "dd" command never work.

Go to details, the source disk have a LOT (really a LOT) of badblocks, so I was suspecting it could be a kind of protection anti-copy, like this ones who you mark a fake badblock and copy a kind of key there to check during boot, however I THINK it's not the case, because this HD really stopped to work :(

Well, what I made:

1) I started the box (using 2 HDs - the original and the destination) with a liveCD and then copied whole disc with "dd" command. Both disks have the same physical geometry, in the front side of the disk is wrote:

Maxtor 20 GB -  geometry = 39703/16/63

I used the command:

dd if=/dev/hda of=/dev/hdc conv=noerror

As all the hardware was exactly the same I expected it to boot, however it do not work. At the end of the clone, if I start machine with original disk it works ok. If I start the machine with the cloned disk, it show "grub screen" but when it start to boot the screen get full black. Strange, not?

The grub have some options ( hdd=ide-scsi vga=791 splash=verbose) not common for me, so I removed some of them and started to get the following error:

Linux-bzImage, setup = 0x1400, size = 0xe147c
Uncompressing Linux...
crc error

-- System halted.

So I googled a little but found not good answers since the hardware was exactly the same.

So I started to analyze the original and cloned image, both have partition like this:


Disk /dev/hda: 20.4 GB, 20490559488 bytes
255 heads, 63 sectors/track, 2491 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1         526     4225063+  83  Linux
/dev/hda2             527         558      257040   82  Linux swap
/dev/hda3             559        4870    34636140   83  Linux

A interesting fact, the partition "/dev/hda1" I can mount as ext2/ext3 without any problems, however partition "/dev/hda3" never mount, it always fail like this:

mount -t ext3 /dev/hda1 /mnt/hda3/
mount
aufs on / type aufs (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
/dev/sda1 on /mnt/hdd type vfat (rw)
/dev/hda1 on /mnt/hda3 type ext3 (rw)
umount /mnt/hda3
mount -t ext3 /dev/hda3 /mnt/hda3/
mount: wrong fs type, bad option, bad superblock on /dev/hda3,
       missing codepage or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
EXT3-fs error (device hda3): ext3_check_descriptors: Block bitmap for group 1 not in group (block 0)!

At dmesg I have:

4>hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
ide0: reset: success
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
ide0: reset: success
A LOT of erros like this....
A LOT of erros like this....
A LOT of erros like this....
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
ide0: reset: success
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x10 { SectorIdNotFound }, LBAsect=78236348, sector=69272078
end_request: I/O error, dev 03:03 (hda), sector 69272078
via-rhine.c:v1.10-LK1.1.19  July-12-2003  Written by Donald Becker
  http://www.scyld.com/network/via-rhine.html
PCI: Setting latency timer of device 00:12.0 to 64
divert: allocating divert_blk for eth0
eth0: VIA VT6102 Rhine-II at 0xee00, 00:1e:90:D8:83:D4, IRQ 10.
eth0: MII PHY found at address 1, status 0x7869 advertising 05e1 Link 45e1.
ip_tables: (C) 2000-2002 Netfilter core team
eth0: Setting full-duplex based on MII #1 link partner capability of 45e1.
SCSI subsystem driver Revision: 1.00
scsi0 : SCSI host adapter emulation for IDE ATAPI devices
ip_tables: (C) 2000-2002 Netfilter core team
ip_tables: (C) 2000-2002 Netfilter core team
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).

For me, nothing relevant, and for you?

as you can see, it's the big partition, I do not believe it's a mistake and this partition is unused....

The "menu.lst" from Grub look like this in the original HD:

#boot=/dev/hda
#password --md5 TheHash
default=0
timeout=1
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Red Hat Linux 9 (2.4.22-i686)
    root (hd0,0)
    kernel /boot/vmlinuz-2.4.22-i686 ro root=/dev/hda1 hdd=ide-scsi vga=791 splash=verbose

The out put of hdparm show us some useful information (which I was not so dumb I could had detected at fdisk output)...

The original disk:

hdparm -g /dev/hda

/dev/hda:
geometry     = 2491/255/63, sectors = 40020624, start = 0


hdparm -i /dev/hda

/dev/hda:

Model=Maxtor 2B020H1, FwRev=WAH21PB0, SerialNo=XXXXXXX
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=off
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=40020624
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes:  pio0 pio1 pio2 pio3 pio4
DMA modes:  mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=yes: disabled (255) WriteCache=enabled
Drive conforms to: ATA/ATAPI-6 T13 1410D revision 0:  1 2 3 4 5 6

hdparm -g /dev/hda

/dev/hda:
geometry      = 39703/16/63, sectors = 40020624, start = 0


hdparm -i /dev/hda

/dev/hda:

Model=Maxtor 2B020H1, FwRev=WAH21PB0, SerialNo=YYYYYYY
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=off
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=40020624
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes:  pio0 pio1 pio2 pio3 pio4
DMA modes:  mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=yes: disabled (255) WriteCache=enabled
Drive conforms to: ATA/ATAPI-6 T13 1410D revision 0:  ATA/ATAPI-1,2,3,4,5,6

* signifies the current active mode

Do you see the difference of geometry? Both disks are equal physically (39703/16/63), however the original shows at Linux as 2491/255/63, sectors = 40020624, while the cloned show 39703/16/63, sectors = 40020624. Very strange, not? The clone of "dd" should not make it be equal?

Looking more and downloading RedHat 9 to install I seen it use a different tool to manage HD, called DiskDruid, I never had used it.... and don't know why, when you create a partition with this tool it change (logically?) the geometry of the disk, which make me believe that is why it's getting different, do you agree?

However I also do not understand why dd doesn't cloned it and changed the geometry, do you understand? It should not clone the place where is the disk geometry too (it's called $Vtable, right?)?

Ahn interesting point, as I seen a lot of badblocks I tried to fix the disk with fsck and I always get this kind of output:

/: |=                                                               |  2.1%   
/: |===                                                             /  4.2%   
/: |====                                                            -  6.4%   
/: |=====                                                           \  8.5%   
/: |=======                                                         | 10.6%   
/: |========                                                        / 12.7%   
/: |==========                                                      - 14.8%   
/: |===========                                                     \ 17.0%   
/: |============                                                    | 19.1%   
/: |==============                                                  / 21.2%   
/: |===============                                                 - 23.3%   
/: |================                                                \ 25.5%   
/: |==================                                              | 27.6%   
/: |===================                                             / 29.7%   
/: |====================                                            - 31.8%   
/: |======================                                          \ 33.9%   
/: |=======================                                         | 36.1%   
/: |========================                                        / 38.2%   
/: |==========================                                      - 40.3%   
/: |===========================                                     \ 42.4%   
/: |=============================                                   | 44.5%   
/: |==============================                                  / 46.7%   
/: |===============================                                 - 48.8%   
/: |=================================                               \ 50.9%   
/: |==================================                              | 53.0%   
/: |===================================                             / 55.2%   
/: |=====================================                           - 57.3%   
/: |======================================                          \ 59.4%   
/: |=======================================                         | 61.5%   
/: |=========================================                       / 63.6%   
/: |==========================================                      - 65.8%   
/: |===========================================                     \ 67.9%   
/: |=============================================                   | 70.0%   
/:                                                                               
Duplicate or bad block in use!
/: Multiply-claimed block(s) in inode 1008: 21140 21268
/: Multiply-claimed block(s) in inode 43894: 94024
/: Multiply-claimed block(s) in inode 59714: 94024
/: Multiply-claimed block(s) in inode 159970: 162719
/: Multiply-claimed block(s) in inode 159976: 174293
/: Multiply-claimed block(s) in inode 160268: 150689 150705
/: Multiply-claimed block(s) in inode 176720: 162719
/: Multiply-claimed block(s) in inode 182890: 174293
/: Multiply-claimed block(s) in inode 192798: 180228
/: Multiply-claimed block(s) in inode 192800: 180284
/: Multiply-claimed block(s) in inode 216113: 290975
/: Multiply-claimed block(s) in inode 216125: 460165 460221
/: Multiply-claimed block(s) in inode 224140: 180228 180284
/: Multiply-claimed block(s) in inode 228972: 460165 460221
/: Multiply-claimed block(s) in inode 255456: 499175
/: Multiply-claimed block(s) in inode 293290: 290975
/: Multiply-claimed block(s) in inode 403215: 349349
/: Multiply-claimed block(s) in inode 407637: 21140 21268
/: Multiply-claimed block(s) in inode 408930: 827359
/: Multiply-claimed block(s) in inode 411372: 150689 150705
/: Multiply-claimed block(s) in inode 411435: 344127 344183 344343 344375 347508 347564 347636 347692 347908 347964 348068 348124 348284 348340 348589 348605 348661 348717 348877 348933 348981 349077 349349
/: Multiply-claimed block(s) in inode 416332: 827359
/: Multiply-claimed block(s) in inode 531944: 344127
/: Multiply-claimed block(s) in inode 531962: 344183
/: Multiply-claimed block(s) in inode 531975: 344343
/: Multiply-claimed block(s) in inode 531977: 344375
/: Multiply-claimed block(s) in inode 532301: 347508 347564
/: Multiply-claimed block(s) in inode 532309: 347636
/: Multiply-claimed block(s) in inode 532319: 347692
/: Multiply-claimed block(s) in inode 532354: 347908
/: Multiply-claimed block(s) in inode 532364: 347964
/: Multiply-claimed block(s) in inode 532402: 348068
/: Multiply-claimed block(s) in inode 532412: 348124
/: Multiply-claimed block(s) in inode 532436: 348284
/: Multiply-claimed block(s) in inode 532438: 348340
/: Multiply-claimed block(s) in inode 532448: 348589
/: Multiply-claimed block(s) in inode 532449: 348605
/: Multiply-claimed block(s) in inode 532454: 348661
/: Multiply-claimed block(s) in inode 532459: 348717
/: Multiply-claimed block(s) in inode 532492: 348877
/: Multiply-claimed block(s) in inode 532499: 348933
/: Multiply-claimed block(s) in inode 532505: 348981
/: Multiply-claimed block(s) in inode 532533: 349077
/: Multiply-claimed block(s) in inode 536690: 499175
/: (There are 44 inodes containing multiply-claimed blocks.)

/: File /usr/local/java/jakarta-tomcat-5.0.28/common/lib/ant.jar (inode #1008, mod time Sun Aug 29 00:02:20 2004)
  has 2 multiply-claimed block(s), shared with 1 file(s):
/:     /usr/lib/locale/locale-archive (inode #407637, mod time Thu Sep 25 19:39:13 2003)
/:

/: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
    (i.e., without -a or -p options)

It's a very interesting case, because even when I choose to fix ALL problems running fsck again manually, when it finish all errors appear again.. strange, not?

I also tried cloning with ddrescue, like:

ddrescure -r 3 -b 64 /dev/hda /dev/hdc rescue.log

But exactly the same happens... :(

I had tried to boot with source disk and mount the cloned and chroot to it and than call grub-install, but it exited unexpected (sigsegv), sometimes at chroot, sometimes at grub-install... very strange... not sure if it's cause the damage at disk or some protection... any ideas?

My last idea, was:

- Create primary "/ mount point" partition and swap using RedHat installer (diskdruid) to have the same installer. - It was done.

- Boot in a live CD and mount both, the do a "cp -a" from the source "/ mount point" to destination, however at this point the HD crashed and I got in serious problems. Cause now we have only the backup machine.

- My next step should try again boot in source disk and mount clone, chroot into it and grub-install, but I had no chance. :(

However the HD which I used as destination of clone was not very good (this model is used and very old) so at the end it stop to work too.

As you can see, I have a big problem and need to do it without mistakes, so I'm asking you experts help. :)

Now I intend to clone it to a machine with the same motherboard but different NEW HD (a new manufacture and model - 160 GB).

However my source disk also is damaged and have a lot of badblocks...

So I ask, what I made wrong? What is the best way to do it? Any tips and hints for me? Any better software to do it?

I need to have maximum security in this process, because if the backup copy stop working, for sure I will be in REAL SERIOUS problems... hehe

If some of you need more information, the output of some command or content of some file just ask, please.

Well, thank you so much and sorry for long post and maybe dummy...

Also, I would like to keep clear that I'm not doing anything illicit, since the company do not exist anymore, we have not for who ask help and also it's important for us.

Thank you all.

Cheers,

Offline

#2 2008-11-13 09:06 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: How to clone a bootable Linux machine?

Wow...  I can't believe you'd have a critical system without regular backups of some
sort...  (Not just the backup duplicate system, but actual hard drive backups of some
kind, as well...)

When you're talking about getting hard drive errors, is that with the old system that
broke down already, or the backup system?  If it's the latter, well you may be in
trouble... ;-/  If it's only the original, then hopefully all you need to do is what you
tried to do with the original...  The simple "dd" should work, I think...  Failing that, the
reinstall followed by "cp -a" approach should work...

But, I'm confused: you say you couldn't mount /dev/hda3, so how did you do the
"cp -a" at all??  Or, were you just copying hda1 at the time?  (And, did you try leaving
off the "-t ext3" to mount, and see if it could auto-guess the FS type?  Maybe it's not
ext3? *shrug*)

As for the odd disk geometry mismatch, I'm not sure...  But, have you checked all
the relevent BIOS settings to make sure they match between the systems?

Sorry I can't be of much help...  Hardware/sysadmin issues like this are really not
my thing...  I try to avoid dealing with hardware issues as much as possible... ;-)

Offline

#3 2008-11-13 10:08 PM

felix
Member
Registered: 2003-12-04
Posts: 171

Re: How to clone a bootable Linux machine?

Hi RobSeace,

First of all, thank you for your fast reply. :)

Yes, backup is a problem, when I arrived at the company the system already exist and it's blackbox systems, no documentation, etc. I and belive it have some kind of protection to do no clone it, because older emplouys spoke if you retrive the HD and put in another exactly hardware it will not boot. Anyway, it's all we have, no only one working machine. And yes, I need to do this job just to create a backup/duplicate this system in the case of this fail.

The hard drive errors are from the old system that broke down, but the backup also is not a new machine, so I suspect it's not 100%.

I already had "dd" the backup system, from a 20GB Maxtor to a BRAND NEW 160GB HD. But it do not work (boot).

I'm a little scarryed of use "cp -a" again, because at this point where all stoped work...

Yes, I can't mount /dev/hda3, so I only made "cp -a" from the hda1, where is the root system. I tried mount /dev/hda3 without "-t ext3" but it fail in the same way. :(

Strange, not?

Yes, the curious about disk geometry, is that both HD are equals, and if I create parition using fdisk they have one geometry, if I use diskdruid it use another geometry, very strange (at last for me).

No problem, you are more than welcome, your feedback is always appreciated.

Some more ideas? Do you know a more professional program to clone bootable partitions/whole HD which deal with geometry, mbr, etc?

Thank you so much.

Regards

Offline

#4 2008-11-13 10:49 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: How to clone a bootable Linux machine?

There are many available disk-cloning apps out there, but I can't personally
recommend any of them...  I've only ever used raw "dd"/"dd_rescue"...  (And, I
suspect most of them work with those at the low-level, anyway...)

But, just a few I know of and have heard good things about: PartImage, Clonezilla, g4u...

Offline

#5 2008-11-15 09:32 AM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: How to clone a bootable Linux machine?

In case of harddisk failure, first thing you do is take it out of the machine, put
it in another (for sure good) one, and copy the harddisk content to a file. Best
thing is to use dd_rescue /dev/broken /path/to/image_backupfile. If it gets "stuck"
on one point, try copying from the other direction with the -r option. If the
partition table is intact, copy each partition separately (starting with the most
important one).

Then do all the rescue attempts on (copies of) the image files instead of the
broken disk. That way the broken disk is stressed as little as possibler. First
step is to run fsck, then you can loop mount the image file.

Copying directly to another disk like you did should work as well, but is less
convenient. Just mount the new disk from a working system and try fixing it
there. But it would be better to do all rescue attempts on copies of the first
image dump, in case you mess anything up. In this case dd(_rescue) from the
new disk to a file on a big hd and fix that. When you're happy with the result,
create a new (bigger) fs, mount -o loop the fixed image file and cp -a all data
to its final destination.

2491/255/63 looks like the correct geometry, but try bypassing this problem by
copying to a file and fixing it there. Run *fdisk and partition the new drive again,
then create a new fs and cp -a all data from image file to the partition. Also
install Grub to the MBR of the new disk (for how see e.g. here).

Offline

#6 2008-11-26 09:01 PM

felix
Member
Registered: 2003-12-04
Posts: 171

Re: How to clone a bootable Linux machine?

Hi RobSeace and i3839,

All worked, the big problem really was the other Hard Disk was completely damaged, so was impossible to recover from it.

With the other HD, i just clonned it with dd, used fsck and reinstalled grub and all worked fine.

Thank you so much for you hot tips. :)

Now we are able to reproduce the system and create a backup.

Now my task is finished, I started to look at the application, how it works, etc. It used a library to create the graphical interface which I never had heard before, it's called "Allegro Datafile".

I seen in this datafile you can put the sounds, images, etc. So I had think, if the company who created this application do not exist anymore (no support, no development, no nothing) I will try to custom the graphical user interface with our logotype and company music.

I had looked at internet but could find any editor for already created datafile.

The unix "file" command return:

data1.dat: Allegro datafile (not packet / autodetect).

Not be packed make me believe it's possible to be edited, right?

Someone have experience with it? Suggest me some tool? some trick how to do it?

Based on system libraries, I believe it use version 4.0 of Allegro Datafile.

Someone know a way to edit resources inside a allegro datafile (.dat)?

Thanks again and sorry for inconvenience.

Chers

Offline

#7 2008-11-26 09:56 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: How to clone a bootable Linux machine?

No idea...  But, google turns up this page...  Maybe it uses this library? *shrug*

Offline

#8 2008-12-18 04:50 PM

felix
Member
Registered: 2003-12-04
Posts: 171

Re: How to clone a bootable Linux machine?

Hi RobSeace,

Yes, it was using this library, and I used a allegro program which worked perfectly. The program is grabber.

http://www.allegro.cc/depot/Grabber/

thank you so much for all help.

regards,

Offline

#9 2009-12-17 10:17 AM

Naguellegange
Guest

Re: How to clone a bootable Linux machine?

Okay... so I got everything running. Time Machine works great on my new EX490 and was able to fully restore a computer today. However, now that I am centralizing all my data the only thing that I am missing is backups of the actual server.

Is there a solution that incrementally backs up changes just like time machine for WHS? I am looking to purchase 2 more drivers or possibly an external drive for off-site backup but I would like to be able to have WHS monitor for file changes every hour just like Time Machine.

Thanks

#10 2013-06-02 07:26 PM

prasad
Guest

Re: How to clone a bootable Linux machine?

hai i have linux red hat system. my probelum error coming (r system:unexpected inconsistency:run fsck manually) how can repair this system. i have a no cd.please help

#11 2013-06-04 11:20 AM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: How to clone a bootable Linux machine?

Prasad, contact Red Hat for support.

(But if it's a different disk than your boot disk you can run fsck manually as root after boot-up.
If it's your root disk then you might be able to run fsck from initrd, if Red Hat installed it there.
If you have a RAID then perhaps look at that first.)

Offline

  • Index
  • » General
  • » How to clone a bootable Linux machine?

Board footer

Powered by FluxBB