Shop OBEX P1 Docs P2 Docs Learn Events
Computer Organization — Parallax Forums

Computer Organization

HumanoidoHumanoido Posts: 5,770
edited 2012-09-21 01:36 in General Discussion
Organizing files on the computer has become difficult. How to arrange so many files, remember names, add new names to the list of diminishing name possibilities, have instant access..?

The number of files may reach a million by the end of the year. OSX 10.6.8 is in use with Spotlight and the Finder Search. Spotlight often returns too many files vaguely related to the search. Search does not work well with files on external drives. What are some ideas for easy organization of many computer files?

Comments

  • Mike GreenMike Green Posts: 23,101
    edited 2012-09-17 22:25
    A lot depends on what you're storing. If your files consist of small pieces of text, you can use a database program to store them along with keywords to help categorize the files. I use DEVONthink and SOHO Notes for this sort of thing. DEVONthink I use mostly for storing internet news clippings and technical articles. SOHO Notes I use mostly for miscellaneous short text files like song lyrics and recipes along with medical documents. Both can handle rich text and media files. You could use Spotlight if you associated keywords with each file as Spotlight Comments. I don't have anywhere near the number of files you have ... maybe 8000 of these short (less than a page) text files. I use both programs for historical reasons. I could use either for all my such files.
  • GadgetmanGadgetman Posts: 2,436
    edited 2012-09-18 04:00
    File organisation is a very vast topic...

    At my office, we have a single share for pictures taken along our roads(two pictures every 20meters, one straight ahead, one for the ditch, then a similar set the other way).
    These pictures now take up the better part of 5TB disk storage...
    (That's for 3 counties, only. They're better quality than Google Streetview, too)
    We have them sorted in folders by County numbers, then year, then road number, and finally, 'road section' (from one major intersection to the next)
    We actually have our own custom picture browser for this that can 'drive' the roads, and also pull out additional data from related files and display at the same time.

    Archival-worthy documents are stored in a database, and can be searched out based on case number, date, user and so forth.
    (Scans of paper documents, Word files, Cad-files, even emails)
    Inserting all the correct data when a document is archived can be a pain, but it needs to be done.
    (We have 'auto-fill' on some fields, though)
    This is typical for how it's done in large offices everywhere.

    Music...
    I use iTunes on my Mac...
    I don't need to know how it's all put together, but it's mostly folders with artists, then subfolders with 'records'. The interlinking into playlists and all the extra data is fitted into an .XML file.

    I have an extensive movie collection.
    (All legal)
    When I ripepd my DVDs I stored them on a networked HDD.
    There I created folders such as '_Anime' and '_Childrens' to sort out two important categories.
    Then I created folders for series of films/TV-shows (Love Hina and Tenchi Muyo in the _Anime folder, Shrek and Ice Age in the _Childrens folder, Adam Adamant, Murder She Wrote, Addams Family and so on at the root level)
    With the special categories having a _ first, they always come at the top of the listing so they're easy to find.
    TV-series and such come alphabetically among the standalone movies.
    Then I add subfolders as and when I see the need for it.
    (Austin Powers got a subfolder even if there's only 3 movies, just because he's got it!)

    The only thing I can say for certain is that you need a plan from the beginning. It may not be perfect, far from it. Everything is better than lumping all the files in one folder.
    If you have some order, even if it's not perfect, it's much easier to correct the deficiencies later...
  • TorTor Posts: 2,010
    edited 2012-09-18 05:32
    I'm not sure how many files there are on my computer.. except that it's at least more than two million files. But it could be much more.
    Searching through all of them would take some time. So the practical idea that works is that either the filename itself is a clue, or it's stored in a directory which name gives a clue. So I search for filenames, or part of filenames, and I use the Unix 'locate' utility for this. At night a database of all files (where some filesystems are excempted from the search) is rebuilt, so using 'locate' is instant. Then I combine keywords for 'locate' with piping through 'grep'. Works well.

    -Tor
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2012-09-18 09:10
    I found that Ubuntu Linux was delightfully quick for several reasons. It just may be that Unix has a better search engine than Windows. But it is also more rigorous in enforcing where users put their personal stuff - so searches don't have to look into areas that are not appropriate.

    I am not sure what OSX does. After all it is a Unix derivative. But I am much happier with LInux than Windows.
  • Heater.Heater. Posts: 21,230
    edited 2012-09-18 12:09
    These are problems that Windows users have but users of normal operating systems do not.
    In my experience Windows search is a pain in the neck as described by Humanoido.
    Just the other day I had to help someone with a classical Windows issue: "I saved my files [from some app. or other] but where are they?"

    In general you can have a lot of files on a computer but, like real life objects, it is up to you to keep them in order or some kind of system. Do you throw all you posessions into one big pile? No, you have projects for this and that. You have geographical and or time slots for this and that and so on.

    Unix tools like "find", "locate", "grep" will help you when you are forgetfull.

    P.S. I remember an artical in a computer magazine in the 1980's or so where the author was exactly asking: With 100MB disk drives how are we ever hoing to find anything? Good grief, how does he cope with todays tera byte drives?

    For me, anything that I have not up loaded to the internet, "cloud" now a days, will vaporize when my hard drive fails. No matter, Google will find it again:)
  • skylightskylight Posts: 1,915
    edited 2012-09-18 14:59
    Heater. wrote: »
    Do you throw all you posessions into one big pile? No, you have projects for this and that. You have geographical and or time slots for this and that and so on.
    Of course.............. yes i do................. ahem(whistle)
  • NWCCTVNWCCTV Posts: 3,629
    edited 2012-09-18 22:28
    I myself create directories for every type of project and also for each client I have. I use seperate hard drives for various items such as Data, Programs, Movies, Music, Pictures etc. and break them down by dates and locations. Seems to work out pretty well.
  • SRLMSRLM Posts: 5,045
    edited 2012-09-19 00:14
    Folders, based on primary use. I have folders for pictures, music, videos, and podcasts. Then everything else is simply project data: either its for a class (in which case it goes into class/<CLASSNAME>/<PROJECTNAME>), or it's a contracting job (in which case it goes in consulting/<CLIENT>).

    The pictures are stored in folders based on when I download the pictures from the camera to the computer. I find that I remember where a picture was taken ("that time I went to that tower in Shanghai"), and from there I can remember when I went ("Oh yeah, it was August of 2010"). It works well for most things. For everything else I use Digikam.

    I don't care about keeping all other documents (data sheets, "cool" pictures from online, executable). All those go into the Downloads folder as a temporary storage. Everything in the folder can be deleted without long term consequences.
  • HumanoidoHumanoido Posts: 5,770
    edited 2012-09-19 02:02
    I think one problem is there are hundreds of thousands of files organized on other external USB drives, from over a decade of work, and when the drive is not online to the computer, its contents are not searched.
  • JordanCClarkJordanCClark Posts: 198
    edited 2012-09-19 02:48
    Maybe a cataloger would be useful. A quick cnet search turned this one up as an example. I'm sure there's sourceforge items out there as well.
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2012-09-19 04:52
    The solution is quite simple..... Think, think, think.

    There is a reason for the directory tree.
  • skylightskylight Posts: 1,915
    edited 2012-09-19 04:58
    JordanCClark wrote: »
    Maybe a cataloger would be useful. A quick cnet search turned this one up as an example. I'm sure there's sourceforge items out there as well.
    The good thing about those types of organiser is that you can free up your hard drive by saving to CD or DVD and read them, just don't lose the discs!
  • JordanCClarkJordanCClark Posts: 198
    edited 2012-09-19 05:15
    Loopy Byteloose wrote: »
    The solution is quite simple..... Think, think, think.

    There is a reason for the directory tree.

    Yes, which does absolutely nothing for not-present discs or unattached drives... think, think, think...:innocent:
    Humanoido wrote: »
    I think one problem is there are hundreds of thousands of files organized on other external USB drives, from over a decade of work, and when the drive is not online to the computer, its contents are not searched.
  • PublisonPublison Posts: 12,366
    edited 2012-09-19 08:05
    I use a couple of these:

    http://www.itechcds.com/html/DC016.html

    Y    eah, I know it's just another external drive, but it comes with pretty good database software that you can search if the unit is offline. I used to use the DC-300 which where CD only, but sold those and upgrade to these units. Internal DVD READ/WRITE. I also have a 1TB external HD, but it takes Picassa a day to organize the photos.
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2012-09-19 09:37
    There was an article in today's local news about a post-doctorial student that has developed a breakthrough in 3D imaging and applied it to increasing the capacity of Blue-ray DVDs to 4 Terrabytes per cubic centimeter (up from something like 1.1Tbytes). It said continuous 8 hour high definiton video is possible on one disk.

    I am sure that will wow some of you and increase the clutter of others.

    Frankly I am having enough trouble with keeping up with life in real time. i don't want an 8 hour DVD.

    And I am more interested in what this guy discovered about 3-D imagining on the nano level. That might relate to super conductors and extremely strong magnetic alloys and much more.
  • Heater.Heater. Posts: 21,230
    edited 2012-09-19 10:39
    Loopy,
    Frankly I am having enough trouble with keeping up with life in real time
    Oh my God, someone who feels like me!

    Speaking as someone why has never owned a VCR (or even a TV for that matter) I have felt that way ever since the invention of the VCR. All of a sudden I saw people collecting shelf fulls of movies and TV shows. Why? I always wondered, when do they think they will ever have the time to watch all that?

    As the years go by the same hording continues but more so, CD, DVD, terabyte hard drives and so on. I know people who if they started playing their video and audio collections now would not live long enough to get to the end.

    My plan for computer organization is simple. Just delete the whole lot after a year or so. If there was anything good on there you would have posted it to the rest of the human race on the net and you can then recover it. If you cannot remeber what you have it may as well not be there. If that seem too harsh then start afresh wit a new HD or even computer, if you one day miss something then go back to the old HD/computer and search for it.

    One day you will drop down dead and all that stuff is gone. It seems unlikely that family and friends are going to scour your bazillions of files to find any interesting stuff before they send the computer to the land fill.
  • SRLMSRLM Posts: 5,045
    edited 2012-09-19 12:33
    Heater. wrote: »
    As the years go by the same hording continues but more so, CD, DVD, terabyte hard drives and so on. I know people who if they started playing their video and audio collections now would not live long enough to get to the end.

    ... It seems unlikely that family and friends are going to scour your bazillions of files to find any interesting stuff before they send the computer to the land fill.

    I like to put on a movie when I'm doing something with my hands, such as building a model or wiring up a circuit. As for the family, all they have to do is look for a folder called "ShowcasePictures": that has the 100 or so best pictures. Everything else can get deleted. :)
  • HumanoidoHumanoido Posts: 5,770
    edited 2012-09-19 13:27
    Heater. wrote: »
    One day you will drop down dead and all that stuff is gone. It seems unlikely that family and friends are going to scour your bazillions of files to find any interesting stuff before they send the computer to the land fill.
    Such a pity. Close to a hundred years of work completely lost. There must be a better way to do this in the future. Perhaps for some reasons it's better to etch your glyphs in stones even if they have less memory space. Egyptian Hieroglyphics could do graphics, and text is easy to scribe. It will certainly last longer compared to the finicky drives and media we use today.
  • PublisonPublison Posts: 12,366
    edited 2012-09-19 13:33
    Humanoido wrote: »
    Such a pity. Close to a hundred years of work completely lost. There must be a better way to do this in the future. Perhaps for some reasons it's better to etch your glyphs in stones even if they have less memory space. Egyptian Hieroglyphics could do graphics, and text is easy to scribe. It will certainly last longer compared to the finicky drives and media we use today.

    How about just write it down on paper? In the USA we still have documents that are three hundred years old.
  • HumanoidoHumanoido Posts: 5,770
    edited 2012-09-19 13:58
    Publison wrote: »
    How about just write it down on paper? In the USA we still have documents that are three hundred years old.
    Perhaps a paperless society is not the best idea as some originally thought.. Certainly we can plant forest after forest of new fast growing tree hybrids that take in CO2 and give off oxygen and provide pulp for paper at the end of their lives. What if the Declaration of Independence was written on a virus eaten crashed hard drive, a melted CD and a lost backup..
  • TorTor Posts: 2,010
    edited 2012-09-21 01:01
    In my experience paper will be thrown out even faster than old computers/computer disks. There are some that are of a mentality which says "Hm, this must be at least a couple of years old, let's bin it". I've lost important one-off (although I didn't know that at the time) documentation at work, by the boxfull. Someone decides to "clean up" the long-term archive room, without understanding what "long-term" means. And then it's gone. So now I scan and copy and triple-copy and upload and whatnot. The electronic copies seem to be a bit safer than paper storage. You'll just have to remember to transfer to some newer media after a while (so that you don't have to do a Pioneer Anomaly tape extraction job later). But people are wiser about this now than in the past.

    And of course all my (now valuable, and, surprisingly, would-be useful) old Personal Computer World mags (from #1) were binned (I did that myself). Regretfully. (Not to mention my old Donald Duck collection from the early sixties.. my mother did that for me, well, I could have sold it now and added to her pension! :))

    Hoarding can be a problem, but I must say that I regret a lot of the stuff I threw away over the years, it's very rare though that I've regretted keeping something. And of course some of the computer-related stuff I threw away ten-fifteen years ago.. disk images and the like.. I didn't know then that later I would find great joy in writing emulators for that old minicomputer hardware, and gnash my teeth about those tape images I threw away. Fortunately found a single copy, but I wish I had the rest.

    -Tor
  • Heater.Heater. Posts: 21,230
    edited 2012-09-21 01:36
    Tor,

    It's heart breaking. I had a huge collection of PCW as well. And then there was the twenty years of Wireless World magazine. All binned in the late 80s when a girl friend got fussy about having a tidy house and I fell for the idea.
Sign In or Register to comment.