Shop OBEX P1 Docs P2 Docs Learn Events
USB Drive Archive Help Please! — Parallax Forums

USB Drive Archive Help Please!

HumanoidoHumanoido Posts: 5,770
edited 2010-03-08 22:22 in General Discussion
I have a pc with Windows XP and several various brand
large capacity USB drives.

When I drag a parallax archive folder over to copy it,
after a few thousand files, it stops and gives the error
that the file name is too long.

I finally figured out that it's the path name that's too
long. That's because I have a half million files.

It's too much to try and shorten the names. It will lose
the identification feature.

When the error comes up, and it stops copy, there is
no way to continue the copy, so I can never copy a
folder completely.

As files increase on my computer, and the need to
copy folders is important, a solution is needed.

There are no backup programs that I have found
which I can trust. They all had some similar problem.

The programs that continued to copy, merely bypassed
the files with the errors, so much information was lost.

Any suggestions?

Thank you in advance,
humanoido

Comments

  • BradCBradC Posts: 2,601
    edited 2010-03-05 00:53
    Format your thumb drive NTFS and investigate rsync.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    You only ever need two tools in life. If it moves and it shouldn't use Duct Tape. If it does not move and it should use WD40.
  • David BDavid B Posts: 592
    edited 2010-03-05 03:15
    Maybe you could zip large groups of the files into zip files, then put the zipfiles onto the USB drive.

    But I'd divide them up into groups instead of trying to zip the entire half million. If you can identify some as files you'd hardly ever access, you could zip them and move them completely off your server; otherwise maybe you could split them up alphabetically or by date or something.
  • kwinnkwinn Posts: 8,697
    edited 2010-03-05 04:19
    I have run into the same problem. At first I manually moved the folders to shorten the path names, which worked but was a lot of work. Later I started to zip them which was also a lot of work until I figured out how to automate it.
  • HumanoidoHumanoido Posts: 5,770
    edited 2010-03-06 00:39
    ZIP sounds like a working solution, but makes me wonder
    if a few years down the road, will it still exist? I would want
    to avoid the scenario where a half million files are zipped and
    no way to unzip.

    humanoido
  • FranklinFranklin Posts: 4,747
    edited 2010-03-06 01:12
    If you're worried about compressing them why not save them to DVD uncompressed?

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    - Stephen
  • HumanoidoHumanoido Posts: 5,770
    edited 2010-03-06 01:38
    Franklin: saving to dvd with no compression is a good idea for lower
    volumes of data. But how many DVDs are needed for a half TeraByte
    of files? My guess is the read and write rates of these drives are
    somewhat slow.

    humanoido
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-03-06 03:51
    Zip exists in so many different incarnations, I doubt that there are any rational grounds to fear for it going extinct.

    -Phil
  • ForrestForrest Posts: 1,341
    edited 2010-03-06 07:56
    If you have 500 GB of files - the solution seems simple. Buy a 500 GB external drive (currently available for around $60 retail), format it NTFS if it's not already in that format and duplicate your entire hard drive.
  • John AbshierJohn Abshier Posts: 1,116
    edited 2010-03-06 15:13
    I remember there is a max number of files for copy or xcopy. I don't remember which. You may want to try SyncToy 2.0 from Microsoft.

    John Abshier

    Post Edited (John Abshier) : 3/6/2010 4:33:56 PM GMT
  • skylightskylight Posts: 1,915
    edited 2010-03-06 17:13
    you could transfer the files to your 3d supercomputer no problems then
  • David BDavid B Posts: 592
    edited 2010-03-06 17:59
    if you want to keep the files available for reference, I'd second that suggestion of buying another hard drive for backup. Or even buying several, and make rotating backups every few months. It's handy to be able to search for old project code anytime you want.

    I've got about a tenth that many files, about 50K files, using about 9 gig of space, that have accumulated over the years, that I keep in a folder on my "E" drive, and Windows has no problem copying the entire folder to an alternate drive with just a drag and drop of the folder. It takes about 10 or 15 minutes to do the backup.

    Why are you getting a "too long filename" error? If you're violating the storage parameters of the destination drive, maybe by nesting files too deep in layers of folders, then maybe all you need to do is flatten out your storage design and you could use your USB drive like you had tried.
  • BradCBradC Posts: 2,601
    edited 2010-03-07 00:58
    rsync or robocopy

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    You only ever need two tools in life. If it moves and it shouldn't use Duct Tape. If it does not move and it should use WD40.
  • VIRANDVIRAND Posts: 656
    edited 2010-03-08 22:22
    It is incredible that you could possibly have and intend to use half a terabyte of
    Parallax microcontroller code in the lifetime of your PC, since ...
    -I doubt so much code has been written for all of these microcontrollers.
    -If the data was all video, it would take you well over a month to watch all of it.
    -Your PC will wear out way before you utilize a half terabyte of microcontroller code.
    -I can't even manage a closet full of backups from the years when the BSoD MTBF rate was once per week,
    and that isn't even half a terabyte.
    -I recommend copying what you are using, hold the originals as backup, until you get a holographic storage drive.

    Where is all that Parallax microcontroller code in your archive collection coming from anyway, I wonder out loud?

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    I should be typing in Spin now.
    Coming soon. My open Propeller Project Pages and favorite links index.
Sign In or Register to comment.