PDA

View Full Version : How to image a new hard drive via USB?



john_s
09-19-2011, 05:04 PM
I need to create an exact image of my brand new laptop hard drive before messing with it but without removing it.
It came with Ubuntu pre-installed and I've heard that there's a straight way to image a hd via USB link to external drive using some Linux tricks.

However, I have a very basic understanding of Ubuntu and I am not even sure where to start.
I'm asking for some advice from Linux experts who are familiar with this.

Thanks,
John

Heater.
09-19-2011, 05:26 PM
Have a look at the "dd" command. Use the man pages or search google. Find out what device your hard drive is with the "mount" command. It will be something like /dev/sda. Plug in your external USB drive and find out what device that becomes, perhaps /dev/sdb.
Then you can copy hd to external drive with dd like so:
sudo dd if=/dev/sda of=/dev/sdb

DANGER if you get the if and of device names mixed up you will write over your hd.

If none of that makes any sence ask some more.

Kevin Wood
09-19-2011, 05:48 PM
Clonezilla should do what you want... http://clonezilla.org/clonezilla-live.php

john_s
09-19-2011, 06:01 PM
I did some search on dd command and they talk about 'duplicating' hd.
Is this the same as what I call an EXACT image?

Will I be able to do the reverse later, i.e. to 'restore' an exact image via USB back to the 'source' hd ?

Heater.
09-19-2011, 06:17 PM
dd makes a byte for byte exact copy. When you are done you can check that this true my using the md5sum command on each drive. The resulting checksums should be the same.

There is one complication I should mention. If you copy the drive that your system is running from of course it can change during the copy which may well upset the results. Possible resulting in non working copy.

Better solution is to boot your system from a live CD like knoppix and then dd the hard drive to the external drive.

Yes you can copy the exact image back. Just reverse the "if" and "of" parameters of the dd command. "in file" and "out file" that is.

john_s
09-19-2011, 06:23 PM
That laptop does NOT have a CD drive - is there a way to go around and boot it from a USB stick?
So here's another problem as I don't even know how to boot a linux pc from a usb stick....

I guess I found it :) http://www.knoppix.net/wiki/Bootable_USB_Key

1st to create a Knoppix CD on another machine, then transfer it to USB and finally use it to boot ...

Martin_H
09-19-2011, 06:44 PM
John, your BIOS setup screen is accessed by hitting a key at initial boot. In that screen there's an option to set boot order so you should be able to see if your BIOS supports USB boot.

Making a bootable Linux flash drive has gotten easy in recent years. Google Pendrive Linux and you should be able to find a utility which can translate a linux ISO to a flash drive image.

john_s
09-19-2011, 08:38 PM
Find out what device your hard drive is with the "mount" command. It will be something like /dev/sda.

You've lost me here.... I typed 'mount' in Terminal and hit 'enter' and it came back with options.... what should I do with this ?

When I plug in my USB drive the screen opens automatically with some kind of 'File Browser' that shows Location: bar with /media/BACK UP inside.

I figure that BACK UP is the original name of my USB drive...

I knew it wasn't gonna be easy :-)

I need a bit more patience and help from you guys as I'm a total newbie with this linux toy...

john_s
09-19-2011, 08:58 PM
John, your BIOS setup screen is accessed by hitting a key at initial boot. In that screen there's an option to set boot order so you should be able to see if your BIOS supports USB boot.


Boot Device Priority in BIOS do not explicitly show USB

- HDD type, model, etc...
- Removable Dev.
- ATAPI CD-ROM
- Disabled

icepuck
09-19-2011, 09:05 PM
since it is ubuntu look in
System->Administration->Disk Utility
that will tell you what your devices are, you can also use it to format/partition just about any usb drive.
then goto
http://www.ubuntu.com/download
and download the same version you have installed and follow the directions for setting up a bootable usb flash drive.
boot up in the "LIVE CD" mode. You can also do the above from the live cd as well.
Look through the Ubuntu forums...
http://ubuntuforums.org/
and...
http://en.wikipedia.org/wiki/Dd_%28Unix%29
You can use dd to save your disk image as a file or copy it over to another drive.
hope this helps
-dan

RDL2004
09-19-2011, 09:29 PM
This should help:

https://help.ubuntu.com/community/Installation/FromUSBStick

Ubuntu runs/installs on pretty much any modern hardware. I almost don't see the need for an image if the computer is new. You may, however want to check and see if this computer is using or requires any "non open source" drivers.

Martin_H
09-20-2011, 12:45 AM
Boot Device Priority in BIOS do not explicitly show USB

- HDD type, model, etc...
- Removable Dev.
- ATAPI CD-ROM
- Disabled

It might be Removable device as the verbiage changes based upon the BIOS manufacturer. If you put that ahead of HDD it will still boot from HDD after the removable device times out or isn't bootable.

RDL2004
09-20-2011, 01:08 AM
On newer machines you can usually just hit a key (such as F12, F10) to access the boot menu directly without having to actually go into the BIOS. Watch the screen as the computer boots and look for such a message, probably at the bottom of the screen when it first starts.

If you go to the Ubuntu.com help page I linked above, it will probably tell you everything you need to know, or point you in the right direction. They are the Ubuntu experts.

Loopy Byteloose
09-20-2011, 01:44 PM
Since you have Ubuntu, there is NO advantage to paying money for a backup image solution - nearly everything for CASH requires Windows, though the older NORTON Ghost would do this.

USE .EXT3 filesystem as EXT4 does not seem to backup right. USE the TAR command for an easier and simpler backup than DD, though both can create complete images (TAR can be followed by GZIP for a compressed backup that is 100% recoverable, thus reducing storage space). There is even a third choice, but I won't bother with it.

The .tar file may be stored on an NTFS formated USB harddisk without any change in data of the image.

Neither DD or TAR will usually back up the hard disk's partition boot information - that is a separate step if you feel you must have it. It is a very small amount of data, 512 bytes I believe.
http://en.wikipedia.org/wiki/Master_boot_record

One of the easiest ways to BU is to use the Ubuntu LiveCD to operate your system WITHOUT the harddisk mounted.
While you can use DD, it may confound you with the line item parameters that are NOT really necessary.

There are other alternatives. The best, is the traditional Tape Archive command that creates a .tar file image. And add to that a .zip file to compress everything. Together, you get a complete and Verifiable image of any partition on the hard disk. You may have to use it multiple times to get all the partitions (maybe NOT as it can copy all the sub-directories). From what I have read, .tar images can allow you to select for individual directories to restore, DD will NOT do that.

DD copies an exact image of each and every block on a hard drive. It is a physical approach. TAR copies an exact image of each and every file on a hard drive. It is a good backup, but not as low level an approach. How exact is exact for you.

One additional backup item is required to get an image of each hard disks, partitioning information. While this is not always necessary, it makes the backup as complete as Norton Ghost or any other commercial backup out there.

Try the Ubuntu website for a Wiki on BACKUP of hard disks. You will have to learn to get around in the /MNT directory to locate your partitions -- though Linux has PARTED for helping you view and manage partition configurations.

And, any LIveCD version of LINUX will work - Knoptik, Puppy Linux, Ubuntu, etc AND all can be made on a USB MemoryStick to be what you call a USB Key.

But, since you have Ubuntu AND it is currently the most popular distribution right now -- the support of an all Ubuntu solution via their own websites, wikis, and forums is likely to be much clearer and less painless.

It is a very good idea to back it up, but if YOU do 'break' Linux before you have it backed up --- you can just use the .tar and gzip to preserve your /HOME directory and overwrite the whole system. Then you can restore just the backup of the /HOME directory. I did this recently as I made the mistake of having an EXT4 file system. This is still not fully supported by many applications, including certain BACKUP services. You should use the EXT3 for the foreseeable future.

And, one added suggestion. If you now have only ONE partition, consider making at least two and maybe three. The second partition would be a /HOME partition to provide a complete separation of your personal data from the / partition. This means makes installing new Ubuntu upgrades much easier as you preserve all your personal data undisturbed by excluding the /HOME partition from future upgrades. And add a /SWAP partition to enhance performance on they memory side. The /SWAP partition will provide additional memory if the DRAM is overloaded.

potatohead
09-20-2011, 02:26 PM
One advantage I can see, is just restoring the factory setup. Takes time to get a Linux all settled in, depending.

What I like to do is get Ubuntu setup on USB Stick. I keep one laying around for a quick boot into Linux as PC admin tool. Works great.

Use this: http://unetbootin.sourceforge.net/ to make your USB key.

The last few times I've cloned hard disks, I've done it this way with good results:

1. Boot your Linux to make sure it works, has driver support, etc... Shut it down.

2. On some laptops, remove the internal drive and mount it external. This is the "source" disk. Mount your "target" disk internal. I know some machines, like a Lenovo Thinkpad won't work well, unless the "target" is on the internal controller. This may vary, and you can always try it once without this step.

3. Once the disks are setup, boot the Linux again for real this time.

4. Run "gpartd", which is the partition manager software. On Ubuntu, it's on the menu, ready to go.

5. Examine the factory disk, taking note of the following:

Number of partitions, their type, and boot flag status.

6. Copy those to the "target" disk, using the copy function in the software.

7. Once they are copied, you can resize the main one to take best advantage of the "target" disk size. If you've not booted the "source" disk, delay this step, until you have.

8. After all copies are done, insure the partition flags are set the same way the "source" disk is.

9. Shut down the Linux.

10. Boot the "target" disk, leaving the source untouched. If you've mounted it internally, this should just be a simple power up operation. If not, then get that done, and boot from it.

11. If successful, complete the setup operations, whatever they are, then consider resizing the partitions to make best use of the "target" disk size.

You could then leave the source untouched. With a Linux, this makes great sense. With Microsoft, it doesn't, because the OS may authenticate during initial boot, and that could fail, if you do it with another disk, just FYI. One boot to the initial state might make sense. Often you get to make your recovery media at that time as well, which also makes sense. Once you've done those things, you can clone the Windows disk with no real worries, and this is something I do all the time, just so it's "Genuine". The recovery media can actually get you back to a factory state, on a new disk, if desired, and supplied. That's the back up, "plan B" option for me.

I like this method because it uses the nice GUI, which will queue your operations before doing them. Good to review before doing a commit to the real disk. I also like it, because many systems have multiple partitions used for various things. These are easy to see and understand.

Finally, some operations take a while. Having the GUI makes it simple to just do it, and walk away. When you return to check in on things, state is easy to see.

All the things I just presented here can be done command line, of course. That's what the simple GUI actually does, and it displays it for you too. (damn cool)

Loopy Byteloose
09-20-2011, 03:02 PM
Excuse me, I have to back down from my dislike of DD. It does do a complete image with the Master Boot Record.

http://www.backuphowto.info/linux-backup-hard-disk-clone-dd

And if use in this manner if can be VERY simple to use.

But it has its place as does the TAR command. Each has advantages. And DD can be used with a LiveCD on a Windows system to avoid paying money for a backup image software.

As you can see, LInux offer more options. Why so? Well, they are free. And then, being based on Unix, it has so many more ways to get things done. Choose whichever tool you feel is best.

With DD, you cannot do partial restores. It is an all or nothing 'Data Dump'.

john_s
09-20-2011, 04:21 PM
I like that 'all or nothing' Alexandre Dumas approach :-)

I keep reading all posts and 'am almost ready to try it - will post my results later.

Thank you folks for your great help.

Loopy Byteloose
09-20-2011, 07:32 PM
Use DD and don't use any compression. That is the most conservative approach. Later when you understand more about Linux, the TAR will be useful.

I have a big project coming up for my 64bit computer, and I will use DD on the 250GB hard disk to a 300gb USB unit. The computer already has Vista in Chinese, but 32bit and Ubuntu Linux in 32bit. I am going to add a 64bit English version of W7 Professional for a triple boot configuration. So starting with an exact image will provide a great level of comfort. It will backup all the partitioning stuff and both the Vista and the Ubuntu as they exist.

I did do some additional research. You also have CPIO and RSYNC usually installed on Ubuntu. And you can use the Synaptic archive installer to get AFIO. There are others, but these are the most commonly relied on.

Loopy Byteloose
09-21-2011, 08:18 AM
BTW, I have been typing all these Linux commands in Upper Case, but Linux is case sensitive (Windows Command Line is NOT)

So, you have to use dd and NOT DD.

john_s
09-22-2011, 01:38 AM
... but Linux is case sensitive (Windows Command Line is NOT) ...

Thanks Loopy, I found it some time ago that it allowed picture.jpg but couldn't even 'see' picture.JPG.

Loopy Byteloose
09-22-2011, 09:04 AM
Like all of us, I have learned a lot by helping out. I am going to use 'dd' in the near future for my 'triple boot' installation. But be aware that these backups are slow and tedious on even the fastest machines. The USB port is a bit of a bottleneck. Make sure everything involved has a good power source or you may fail to get a good image.

On my EEEpc 4G, I used a Python program that someone created for backup and recovery that was excellent - it either used 'cpio' or 'tar'. I think it used 'tar' and after making a file, it then used the -diff option to verify that the file system and the .tar.gzip file was exactly the same. That may be useful info for future backups with compression.

With the EEEpc and just a 4gbyte image, I had to wait about 25 minutes for creating an image or reinstalling one. If you do partitioning as well, this can take huge amounts of time as it will verify every sector of the disc.

Above all, join the Ubuntu Forum as they are excellent at helping out.

john_s
09-22-2011, 04:57 PM
You've mentioned eee machines as that's exactly what I have. One is eee 1001P that has a 250 GB hd and the other is.... eeePC 701 4GB :)
The 701 I got only recently as a play with linux machine. Now I'm even more interested in finding out how did you image that 701 and never mind the time it takes.
I have plans to play with XP and ubuntu installed on that 701 anyways, so I need an image of its original 4GB to restore it back to its original Xandros Linux after I'm done.

I only recently subscribed to Ubuntu Forum, and yes - folks there are excellent support!

Thanks,
John

Loopy Byteloose
09-23-2011, 04:43 PM
Well, the EEEpc 4G is a whole different deal as I got software that was intended specifically to back up the Xandros image (which has a weird partitioning scheme intended to protect the solid-state harddisk from wear), but I also used it to load Ubuntu notebook images without the protective software in one big partition. The initial software must be created in Xandros, but then it doesn't care what OS it backs up. And it can be directed to store the image on another USBkey or SDcard. As I said, that was written in Python and I may have to dig at the EEEpc Users Forum to locate a download. I think they may still have a Wiki on it, but I get the feeling that with so much time passed it may have gotten buried.

These 250gByte hard disks are a bit ridiculous to image. Way too big and I never fill them. I am dreading the time it might take to do a 'dd'. Usually I just do a TAR for the /home and include gzip. That has all my personal stuff. The rest of the OS can be repaired with a LiveCD and reinstall.

I will try to report back with specific info. You could try to 'dd' the EEEpc 4G to a larger than 4Gbyte USBstick or SDcard. It is supposed to work fine. But I suppose you want the menu driven software.

Loopy Byteloose
09-23-2011, 04:58 PM
http://wiki.eeeuser.com/backup_procedure_for_eee_with_xandros

This is what I used with the EEEpc 4G. I miss it, I wore mine out after 3 years of heavy use and moved on to a Toshiba NB250 which isn't as good as an Asus. Next time, I will go back to Asus.

john_s
09-23-2011, 06:09 PM
... I wore mine out after 3 years of heavy use ...

Will try it this weekend... Did you actually wear out the 4GB mobo memory?

Thanks,
John

Loopy Byteloose
09-24-2011, 06:04 AM
No, actually what I wore out was the wall wart, but I initially thought I had worn out the power supply circuits on the motherboard. I decided that it was beyond repair and bought the Toshiba rather than seek help from Asus here in Taiwan. I was getting a lot of very strange battery charging behavior. I gave the machine to a friend that that speaks Chinese and he took it into their service department and paid $45USD for a new wall wart plus service and it is now back to running fine. He loves it as it is so fast and stable. And that is what I miss the most, the solid-state hard disk boots lightning quick and you never have to worry about it being damaged by a severe bump. It is a great travel machine. I took it to Thailand for three weeks with a USBkey backup in case of trouble. The small charger is nice too. (Toshiba has a huge one.)

But since I gave it too him broken, I can't really ask for it back.

I've been thinking about the backup software on the USBkey. While it requires Xandros to make the USBkey (likely due to the right version of Python), it will likely boot any computer (Windows, any Linux distro, OSX) and make a backup image to a USB port or SDcard. But as I said, big disks are very tedious projects, especially with a USB port involved. It doe have an image verify routine as well, but that is nearly as slow as the backup.

Loopy Byteloose
09-24-2011, 07:55 AM
Just yet another afterthought.

'dd' has TWO destination modes - one will actually clone the hard disk AND the other writes an image to file. Be careful of the example code as many assume that the image will end up in your /home directory (by using the ~/ notation).

You need to make sure that your USB disk drive path is specified AND that you have provided a good file name. Otherwise, you might accidentally clone the hard drive. That may be of use to you. But it could mean that you would want to just start over after wasting a bit of effort.

That ~/ notation is not something that a new Linux user is likely to fully understand.

Loopy Byteloose
09-24-2011, 08:58 AM
More things came to mind.

I came home and located my USBkey with the Xandros Linux SimpleBU software installed and explored using it on my Toshiba. The only snag is the video is specifically set up in Grub for the EEEpc 4G. So you have to work throug a screen that traps the wrong video output. I used an 80x50 choice, but I thing quite a few others may work.

So what is this little Backup software about. It seems neither fish nor fowl. The program creates a limited Linux environment in Ram specifically for making backup images and restoring them. I uses Python 2.4 and not the conventional Bash shell or BusyBox programs. The image is gzipped, but usually not a TAR file if it is a complete disk image. The software creates a .disk.gzip file.

So it seems quite useful on any system that boots from a USB port. It eliminates the need for a LiveCD and a using 'dd'. It walks you through a menu system to 'backup', to 'restore', and to 'verify'. So it is a quite handy means of disk imaging if you have it. I am quite thankful that I save it after I gave away the EEEpc.

Now I am wondering if the original software can be manipulated to install from any Linux distribution. That is yet another project. Mostly, I was concerned that 'dd' might frustrate you and this would provide a good alternative. But the EEEpc software won't directly clone hard disks.

Since there is a mention in the documentation that these .disk.gzip images can be manually restored with a 'dd' command, I suspect that the file is generically a 'dd' image with compression.

john_s
09-24-2011, 05:13 PM
Loopy, I also have an original "Support DVD Rev.1.3" A023. Any thoughts if it might help me in case of a failure?
I kind of guess that it might do just a 'factory default' eee setup/restore while yours makes a perfect image of the actual 701 including all user's made apps, dirs and subs.

How much different is it from what you have described?

Loopy Byteloose
09-25-2011, 05:04 AM
Hi John,
First of all, you might want to make an image with 'dd' of what you have on the EEEpc if you are dumping it. That Knoppic or Puppy Linux USBkey will allow that to be done. I am guessing you may have XP or something else installed.

If you can just run the DVD and install the factory default Xandros, then you can eventually make a 'Simple EEEpc backup' USBkey. Apparently some have complained that the SDcards are more sensitive to booting failures due to poorer electrical contact, so avoid cards.

Depending on your verison of the Xandros DVD, you may get a Chinese install. The EEEuser.com forum can explain how to get to the English version. The four partitions are intended to do several things. One big one is for the original Xandros image, the second is for data and changes to the image, and the other two have something to do with buffering the DVD (they are NTFS partitions). If you have trouble installing, you might have to created four partitions according to spec with Knoppic or Puppy Linux by running GParted, but my last experience with my old EEEpc-4g is that I simply ran the DVD and loaded Xandros into before giving it to my friend (He didn't want Ubuntu Linux and Xandros is really great for kids).

I took another look at the code for installation of the Simple EEEpcBU and the file is a .shark.sh extension. I am not sure what .shark.sh is and cannot make it work outside of Xandros. Searches reveal next to nothing about it. But once made, the USBkey will work to do backups on any USB bootable machine. This .shark.sh seems to be a way of making sure you have Xandros in order to get the free BU software, XP will not run it and Ubuntu Linux will not run it. I cannot even open it as a text file.

Failures are very well-supported by the EEEuser.com forum. But I doubt if you will have one unless the DVD is simply defective. There is always a risk with this low level stuff, but the vast majority of the outcomes are good. Nonetheless, good constant power is a must. Make sure the battery is full and the AC is also plugged in.

My own experience with Ubuntu Linux on the EEEpc was that one partition was adequate. One might consider some modifications to reduce the amount of writing to the solid-state hard disk. The first one is to use EXT2 file system, not the EXT3. EXT2 does not journalize, so it may require running a recovery procedure if you have an abrupt shut down. But that procedure is pretty much done by Grub at startup if required - it is a mindless detail. Other choices are to reduce the amount of writing to logs, not providing a SWAP partition, and upgraded the DRAM in the machine so that you do less writes to the solid-state hard disk. The bottom line here is that nobody really knows if and when the solid-state hard disk will wear out. I initially worried about it a great deal, but since my 701-4g had the chips soldered in place, I decided to forget about worrying. Some EEEpcs have socketed solid-state hard disks and can be replaced when the need arises.

BTW, I did not particularly like the Xandros Linux. As I mentioned it is rather 'kiddie' oriented. And Xandros really is trying to sell you support for any alternative. You may work with it for a few days and decide to just move on to Ubuntu netbook remix or even standard Ubuntu.

The biggest complaints about the EEEpc 701-4g is the small keyboard. It takes getting used to. After that, the small screen requires using an 'ALT-F7" to move sections of windows to where you can select features. Both issues are not serious.

john_s
09-25-2011, 06:05 AM
From what I found so far this will be a long way before I change Xandros into another flavour of Linux on that 701 machine. Today I 'discovered' terminal (or console as they call it). As I used to play with MS software it's a whole new experience to find 'terminal' that has nothing to do with a COM port :-) Another surprise came when in a process of learning while searching I found a way to place a missing START icon at the bottom left corner.

What I'd like to change is a look of those screens with big icons. Is there a way to group some applications that I plan to use and glue them inside a single Favorite screen instead?

Another feature I need is to be able to zoom text inside any of those screens as some of them require a magnifying glass (at least I need one). Any hint on that? Is there a shortcut that I can use every time I need a bigger font (in terminal particularly)?

Thanks,
John

Loopy Byteloose
09-25-2011, 09:58 AM
Hi John,
Terminal is just another name for the Command Line Prompt in Windows; some Linux distros call it Console.
But you have a lot more resources in your command path - thousands of commands (Try a Tab+Tab from the Terminal to get a full list.) Also there is Help, Man for manuals, and Info for more info.

You can change Xandros' visual presentation. Just look in the Wikis at EEEuser.com. There is really a wealth of information there for the 701-4g as that was its starting point. The later EEEpcs have gotten more diverse and support for them is not as specific. Also, some of the people that really understood the platform may have moved on and are no longer participating.

I have a copy of "How Linux Works - what every superuser should know" by Brian Ward. I find it an excellent starting point that you likely won't regret buying. I have a lot of other stuff - mostly from O'Reilly, but this one presents it in a sensible and informative way.

At some point, you will begin to understand Xandros and that will likely inspire you to move on. The first 2G partition is untouchable write only, so the only way to add or update programs is to fill the other 2G with updates while leaving the original in place. So space runs out. And you have to rely on an Xandros repository to add things. That seems rather limited when compared to the Ubuntu repositories. Trying to load fron non-Xandros repositories can be daunting from within Xandros.

The really neat thing is that if you greatly mess up the OS, just reboot with F9 and your are purged back to the original image of Xandros. That is great if you let a kid use the machine without supervision.

BTW, from Terminal, you do have to call up Telnet or Putty to get to an RS-232 port.