Shop OBEX P1 Docs P2 Docs Learn Events
C: SimpleIDE... Directory Listing of SD Card Files - Page 2 — Parallax Forums

C: SimpleIDE... Directory Listing of SD Card Files

2

Comments

  • Dave HeinDave Hein Posts: 6,347
    edited 2015-05-02 05:50
    There's no mystery about filename versus pathname. They are just different labels for the same thing. Both of them are evaluated starting from the current working directory. You can pass a filename/pathname that looks like "../directory/file1", or "directory/file2" or even "/directory/file3" or "directory/subdir/file4". In the last example the original program failed it was specifying "subdir" instead of "directory/subdir". I could have done a combination of calling chdir and opendir, which would have been a different way to solve the problem.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-02 16:26
    I still do not have it working 100%, but I keep getting closer.

    This project has a couple of useful functions, but the ones I really want to work are still having some problems. More specifically, the contents of an open directory are seperated into two different arrays, being files and directories. After seperation, the arrays are then sorted alphabetically, followed by first printing the directories, with a <DIR> prefix, and thereafter a file listing.

    I am sure there are a variety of problems, but like I said, I am getting there.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-03 05:28
    Well I got the sorting sorted out :)

    I have not verified it, but apparently, SD card files are probably sorted according to a time stamp, instead of by type (file or directory) and alphabetically. Well this project sorts by type and alphabetically. Additionally, this project is an SD card file and directory access wrapper, with the following members:
    int init_fs();
    int create_dir(char *path);
    int delete_dir(char *path);
    FILE *open_file(char *filename, char *mode);
    int close_file(FILE *file_pointer);
    size_t write_file(void *ptr, size_t size, size_t count, FILE *file_pointer);
    int delete_file(char *filename);
    char *get_curr_dir(char *buf, int size);
    int set_curr_dir(char *path);
    char *path_add_slash(char *path);
    bool path_append(char *path, char *more);
    void sort_files();
    bool compare_swap_files(int pos);
    void sort_dirs();
    bool compare_swap_dirs(int pos);
    int index_dir_entries(char *dir_path, bool sort);
    void print_dir_entries();
    

    Within the main function, index_dir_entries is called on two different occassions, one with the second parameter being true and the other with the second parameter being false. Sorting of the files and directories is controlled by the second parameter.

    This project creates and deletes a total of 52 directories and a total of 52 files. If something wierd should happen, these files and/or directories can be deleted with the appropriate functions.

    EDIT: Please be patient for the print times. There is a short lull in between the first print job and the second print job.
    EDIT: This project is setup for the Propeller BOE. If you use a different board, please set the appropriate pins in file_services.c
    EDIT: Additionally, this project is currently set to read a maximum of 100 file and 100 directories per execution. If you access a directory which has more that 100 files or more than 100 directories, you must increase the size of MAX_FILES and/or MAX_DIRS within file_services.c. And furthermore, this project is setup for 8.3 short file names.
  • Dave HeinDave Hein Posts: 6,347
    edited 2015-05-03 09:19
    idbruce wrote: »
    I have not verified it, but apparently, SD card files are probably sorted according to a time stamp, instead of by type (file or directory) and alphabetically.
    readdir does not sort the SD card files. It just returns the next file in the directory.
  • potatoheadpotatohead Posts: 10,261
    edited 2015-05-03 11:13
    Yes, at a basic level, it's just the order in which things get created and written. Higher level functions can present that information in various ways, or not.

    Just go and look at the raw directory entries, or take a look at how FAT works: http://www.tavi.co.uk/phobos/fat.html
  • idbruceidbruce Posts: 6,197
    edited 2015-05-03 12:29
    Dave and Potatohead

    I wouldn't have had to write sorting functions, if it did the sorting for me. I am just saying....

    Now for the tricky part, which will be displaying the sorted directories and files to an LCD display, and making the directories navigable through the use of pushbuttons. Meanwhile tracking the current directory, but also allowing reversal within the directory structure.

    I still have to write the function, to strip the current directory back one level, to allow reversing.
  • potatoheadpotatohead Posts: 10,261
    edited 2015-05-03 12:40
    To be more precise, you had to write sorting functions because you wanted them sorted. :)

    They don't have to be. Just saying... :)
  • idbruceidbruce Posts: 6,197
    edited 2015-05-03 12:51
    Potatohead
    To be more precise, you had to write sorting functions because you wanted them sorted.

    They don't have to be. Just saying...

    LOL... I stand corrected. Yes, I did want them sorted. Just my own preference. I believe in the long run, it will be easier to navigate to the desired directory and file. I plan to organize 3D printable objects by category.
  • potatoheadpotatohead Posts: 10,261
    edited 2015-05-03 13:14
    Yeah. Given the small memory, having basics there is important. Do too much, and it's not as generally useful.

    Well, it appears you have it all sorted now.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-03 13:30
    Potatohead
    Yeah. Given the small memory, having basics there is important. Do too much, and it's not as generally useful.

    I agree, the project grew to a good size.

    Considering that the SD code by itself eats up a lot of memory, it tends to make things a bit cramped. When it comes time to implement the sorting functions in the 3D printer firmware, I may have to dump the rest of the wrapped functions and go straight with the original C functions.

    However for small logging projects or setting up a file structure on an SD card, it may be useful to someone.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-04 20:08
    I have been toying around with and modifying the code for this project, in an attempt to shrink the build size.

    I truly don't suppose that it would do any good to tell you about all the changes, but I will say this, the build size will largely depend upon your needs. Code can be added, subtracted, or modified to suit your needs. All of the modifications that I made, pertained directly to the sorting and display of directories and files. If you intend to keep and use the sorting code, there are a few simple rules that should be maintained, unless of course you change some variables.

    *File paths should have a maximum depth level of 8, using 8.3 short file names.
    *Limit file names to include 0-9, a-z, A-Z, _, and ..
    *A maximum limit of 100 sub-directories and/or files per directory.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-05 08:14
    In Post #1 of this thread, I stated:
    Basically, I want to list all the files and/or directories on an LCD display, so that a user can navigate this list, through the use of pushbuttons, to select a desired file.

    With this thought in mind, I have slowly been progressing towards this goal. As of this morning, I have added two more important functions to the project, which are:
      char *path_prev_level(char *path);
      void up_dir_level();
    These two functions are important, because they allow navigation to previous directories, and eventually, a single pushbutton will navigate from the current directory, to the parent directory, until the root is reached, with one button press per directory navigation.

    Besides working on this code, I now have my LCD display attached to the Prop BOE, and have been toying around with that a bit. I still need to do the bread boarding for the required pushbuttons, to navigate the file system. The pushbuttons will likely be comprised of the following:
      scroll file list up the LCD display
      scroll file list down the LCD display
      select file or folder
      confirm selection
      reset LCD display to current directory
      go to parent directory
    On second thought, I believe I will just add a 10 pin header to the Prop BOE and use my user interface board :) That will make it easy :)

    Anyhow, after this example upload, the code in the main file, will start heading in the direction of my main goal. I am simply providing a copy of the existing code, which includes the two new functions, before I move forward.

    Please note: This code has not been tested 100%, so it may contain slight errors.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 04:09
    For those that may be interested.....

    I finally got tired of "." and ".." showing up in my file and directory listings, so I modified the index_dir_entries function to exclude these items. The new code for this function is as follows:
    int index_dir_entries(char *dir_path)
    {
        dir_entries_count = -1;
    
        DIR *sd_dir;
        struct dirent *dir_entry;
    
        sd_dir = opendir(dir_path);
    
        if(sd_dir != NULL)
        {
            while((dir_entry = readdir(sd_dir)))
            {
                if((strcmp(dir_entry->d_name, ".") != 0) && 
                    (strcmp(dir_entry->d_name, "..") != 0))
                {
                    dir_entries_count++;
                
                    if(dir_entry->attr == ATTR_DIRECTORY)
                    {
                        strcpy(dir_entries[dir_entries_count], "(DIR)");
                        strcat(dir_entries[dir_entries_count], dir_entry->d_name);
                    }
                    else
                    {
                        strcpy(dir_entries[dir_entries_count], dir_entry->d_name);
                    }
                }                                
            }        
    
            closedir(sd_dir);
            
            sort();
        }
        else
        {
            print("opendir could not open %s", dir_path);
        }        
    
        return 0;
    }
    
  • Heater.Heater. Posts: 21,230
    edited 2015-05-06 05:29
    Why would you get tired of "." and ".."?

    They are great, select them from whatever GUI you have and you get to where you want to be. No "up" button required.

    Well OK, "." is not so useful in that context.

    Like this on a web directory listing:
    attachment.php?attachmentid=114081&d=1430915362
    696 x 307 - 24K
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 05:58
    Heater

    I am glad you provided the pic, because I never thought of the ".." in that context, while coding. In fact, I never actually got around to finding out what the "." and ".." actually was. I just knew that it had to be taken into consideration when performing recursive searching.

    So in the context that you provide, ".." is the parent directory, correct?

    So what is the "."?
  • Heater.Heater. Posts: 21,230
    edited 2015-05-06 06:39
    idbruce,

    Let's say your current directory is "/x/y/z"

    Then the ls (or dir) command will get you a list of files and sub-directories in "/x/y/z"

    "ls .." will get you a list of files and sub-directories in /x/y. The directory above your current directory.

    So what is the point of "."?

    In a normal Unix system you cannot run an executable program, say "myprog", that is in the current directory by simply typing "myprog".

    Why? Because the OS will only look for commands to run in the directories listed in the PATH environment variable. The current directory is probably not on the PATH.

    So if you want to run "myprog" you need to give the full path "/x/y/z/myprog".

    Well, "." is short hand for the current directory so you only need to type "./myprog"

    All of this is a security thing.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 06:51
    Heater

    Thanks for the explanation. I will look into this a little more, just to see how Windows make use of these dots. I know they are there for some reason :) I just never seen them being utilized, besides checking for their exisstence.
  • Heater.Heater. Posts: 21,230
    edited 2015-05-06 07:12
    There is a certain logical elegance about "." and "..".

    In a good old adventure game you could go "N", "W", "S" or "E". Those were you possible moves, relative to where you are.

    In a directory structure you can move down to one of many sub-directories. The other way out is up, relatively: ".."

    But here is "."

    These dots are quite often useful when scripting things. You don't need to know the absolute path to the directory you are on. Only where things are relatively.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 07:21
    These dots are quite often useful when scripting things. You don't need to know the absolute path to the directory you are on. Only where things are relatively.

    Except for occassional use in PERL scripts, I never used them. For some odd reason, I have always entered the full path, when possible. However, since you mention it, I have seen them used quite a bit, for relative paths in web programming.
  • Heater.Heater. Posts: 21,230
    edited 2015-05-06 08:12
    idbruce,
    I have always entered the full path,...
    Exactly, every such use is a bug in waiting. When things get moved everything breaks. Which as you notice web programmers are well aware of.

    Remember all those Windows applications that had to be installed to C:whatever? Select some other drive or directory at installation time and nothing worked?
  • potatoheadpotatohead Posts: 10,261
    edited 2015-05-06 08:45
    Another use for the "." dot is when one wants to make a symbolic link of a directory to another place in the filesystem, or you want to make links in a relative way, or you want a linked directory to appear as a real directory to the command, not just return the link information.

    http://teaching.idallen.com/cst8207/13w/notes/460_symbolic_links.html

    (not sure Windows supports all the linking UNIX does)

    It's not so important to include the single dot, though doing so does allow for a filesystem refresh of the current directory without having to code or provide an interface for that capability.

    Say a person copies a file while another is in the GUI selecting files. One either needs to code in support for file notification, which would trigger a redraw of the file list to show the recently added file, or provide a refresh button, or have the user back out of the dialog, or directory depending on the read policy. Some dialogs read it all, and don't refresh, until requested. Others read directories on entry.

    If you code for the latter read directory on entry, than a single dot means "enter current directory" and that would then display the new file, or changes to the files since the last scan. The use case would be "user does not see expected file" The work around for not having this is usually either to go up one directory, then back down, or exit file selection and return to force a reread of everything.

    That little dot is kind of handy in this context, but it's not so important on a Propeller, given there typically isn't, though there could be, multi-user / process use of the SD card, but that's what the thing is for and what it does.

    Now that I think about it, one could cheat the prop by removing the SD card, adding a file, then putting it back without actually doing anything on the Propeller. In that scenario, just picking that dot would actually return file changes, which would be expected, though poor practice, behavior.
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 09:12
    Exactly, every such use is a bug in waiting. When things get moved everything breaks.

    I guess I should not have said that, because it wasn't true. I am not sure what the standards are now, but way back when, to receive Windows compatible stamp of approval, certain criteria had to be met, and there was some pretty strict criteria. If I remember correctly, all program files had to reside in the programs parent directory of the Program Files folderand Windows specified location, and registry settings were somewhat similar. Anyhow, instead of hardcoding C:/..../...../etc You would use a function like this instead, to obtain roots, and this function could also be used at runtime, and any additional appendage could be added to these roots to obtain the final path:
    BOOL CFileServices::InitializeDirPaths_FS()
    {
    	BOOL bDirIniResult = TRUE;
    
    	HKEY hKey;
    
    	DWORD lpcbData;	
    
    	GetWindowsDirectory(szWindowsDir, MAX_PATH);
    
    	lstrcpyn(szMainDrive, (LPCTSTR)szWindowsDir, 3);
    
    	RegOpenKeyEx (HKEY_LOCAL_MACHINE, "SOFTWARE\\Microsoft\\Windows\\CurrentVersion",
    		0, KEY_QUERY_VALUE, &hKey);
    
    	RegQueryValueEx (hKey, "ProgramFilesDir", NULL,
    		NULL, NULL, &lpcbData);
    
    	RegQueryValueEx (hKey, "ProgramFilesDir", NULL,
    		NULL, (LPBYTE)(LPTSTR)szProgramFilesDir, &lpcbData);
    
    	RegCloseKey (hKey);	
    	
    	//  This is only a few of the available folders
    	// and the following path functions are just a few
    	// of those which are listed in shlwapi.h
    	SHGetSpecialFolderPath(NULL, szProgramsDir, CSIDL_PROGRAMS, 0);	
    	SHGetSpecialFolderPath(NULL, szDesktopDir, CSIDL_DESKTOPDIRECTORY, 0);	
    	SHGetSpecialFolderPath(NULL, szStartupDir, CSIDL_STARTUP, 0);	
    	SHGetSpecialFolderPath(NULL, szStartMenuDir, CSIDL_STARTMENU, 0);	
    	SHGetSpecialFolderPath(NULL, szMyDocumentsDir, CSIDL_PERSONAL, 0);	
    	SHGetSpecialFolderPath(NULL, szFavoritesDir, CSIDL_FAVORITES, 0);
    
    	if(strcmp(szWindowsDir, _T("")) == 0 || 
    		strcmp(szMainDrive, _T("")) == 0 ||
    		strcmp(szProgramFilesDir, _T("")) == 0 ||
    		strcmp(szProgramsDir, _T("")) == 0 ||
    		strcmp(szDesktopDir, _T("")) == 0 ||
    		strcmp(szStartupDir, _T("")) == 0 ||
    		strcmp(szStartMenuDir, _T("")) == 0 ||
    		strcmp(szMyDocumentsDir, _T("")) == 0 ||
    		strcmp(szFavoritesDir, _T("")) == 0)
    	{
    		bDirIniResult = FALSE;
    	}
    
    	return bDirIniResult;
    }
    
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 09:19
    potaohead
    Say a person copies a file while another is in the GUI selecting files. One either needs to code in support for file notification, which would trigger a redraw of the file list to show the recently added file, or provide a refresh button, or have the user back out of the dialog, or directory depending on the read policy. Some dialogs read it all, and don't refresh, until requested. Others read directories on entry.

    If you code for the latter read directory on entry, than a single dot means "enter current directory" and that would then display the new file, or changes to the files since the last scan. The use case would be "user does not see expected file" The work around for not having this is usually either to go up one directory, then back down, or exit file selection and return to force a reread of everything.

    I was fairly certain that the Windows shell did auto updates, but you made me look :)
  • Heater.Heater. Posts: 21,230
    edited 2015-05-06 09:22
    idbruce,

    Please don't ask me to look at the legacy Windows code. Such monstrous images cause me sleepless nights!
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 09:33
    Please don't ask me to look at the legacy Windows code. Such monstrous images cause me sleepless nights!

    LOL..... I was just correcting my misleading statement. :) I don't want you to lose any sleep over it :)
  • idbruceidbruce Posts: 6,197
    edited 2015-05-06 09:35
    On an upbeat note... I now have SD files showing up on the LCD... although not much versatility yet.
  • potatoheadpotatohead Posts: 10,261
    edited 2015-05-06 09:47
    I was fairly certain that the Windows shell did auto updates, but you made me look

    Well, there is a lot more than windows out there. Yes, it does today, but not all dialog tool kits do, including those that one might find on a windows machine. And it's sometimes nice to know why and how things come to exist.

    ...and if somebody does run your stuff, and does cheat with the SD card, and I might knowing how a Prop does things, it would work just fine!

    That said, no need for the ".", but now you know something about that dot. Never a bad thing, right?

    FWIW, back in the day, seeing that stamp of approval meant it was officially broken. :) I was running a lot of nice IRIX, UNIX, Linux systems. My only windows box was in a corner, with some desktop remote software on it. The goal was to turn it off, and I eventually did for a number of years. Good times, Back then, windows was a clunker. It's much improved today, but at the time of those standards? Run!
  • idbruceidbruce Posts: 6,197
    edited 2015-05-08 05:30
    First off, let me just say that this has been a very fun and interesting project, involving the Prop BOE, a #27979 Parallax Serial LCD, six pushbuttons, two LEDS, and a 2GB micro SD card. I really like the LCD...., nice product Parallax. The LCD is a backlit, 4 X 20 character display, with a piezospeaker.

    Anyhow, the goal of this project is to navigate the directory structure of an SD card, through the use of pushbuttons, with the LEDs and the LCD being visual indicators of success or failure, and with the piezospeaker also indicating error, ultimately resulting in the selection of a file. I am very close to achieving this goal, but now I have developed a problem on the home stretch. And the home stretch is changing to a different directory and selecting a file.

    I am now at a point, where the pushbuttons scroll the display of files and folders rather nicely, with the proper sound and visual indicator code all in place. In other words, for the first selection of files and directories, everything works perfectly, but when changing directories, and listing those files and sub-directories, that is where things start to go haywire. I am really not sure where the problem is, but I have a few sneaking suspicions, which I will now share with you.

    char dir_entries[MAX_ENTRIES][18]; is used for listing and tracking directory entries, which includes both files and directories. Whenever a directory has been selected for navigation, this array must be wiped clean, so that a new list of files and directories can be entered. To clear this data, I am using the following two functions. I should mention that I believe this code is working
    void empty_dir_entries()
    {
        for(int i = 0; i < MAX_ENTRIES; i++)
        {
            clear_entry_str(dir_entries[i]);
        }
    }
    
    void clear_entry_str(char *entry)
    {
        int i = 0;
    
        while(entry[i] != '\0')
        {
            entry[i] = '\0';
            i++;
        }
    }
    

    Here is another potential problem. To assist with navigation, I have been setting the cursor to the first position of each LCD line, as directed by button presses and programming code. To provide clarity of current cursor position, all strings have been shifted one space to the right, with the following format " string". Additionally, as mentioned earlier, directories and files are sorted by type and alphabetically, with the directories proceeding files in the listing. To assist with the sorting and listing process, each directory listing is proceeded with "(DIR)". These cursor spaces and directory prefixes are stored in the array above, along with the file and directory listing. When a file or directory has been selected, the proceeding cursor space and if present "(DIR)" prefix, must be removed from the string, held in the array above, to ensure proper navigation. To remove the cursor space and potential "(DIR)" prefix, I am using the following code, and I believe this is where my problem lies. I have said it before..... MFC spoiled me with the CString class :)
    void on_confirm_sel()
    {
        if(entry_selected == true)
        {
            entry_selected = false;
    
            char *temp = NULL;
        
            if(lcd_cur_line == 0)
            {
                strcpy(temp, dir_entries[last_visible_entry - 3]);
            }
            else if(lcd_cur_line == 1)
            {
                strcpy(temp, dir_entries[last_visible_entry - 2]);
            }
            else if(lcd_cur_line == 2)
            {
                strcpy(temp, dir_entries[last_visible_entry - 1]);
            }
            else if(lcd_cur_line == 3)
            {
                strcpy(temp, dir_entries[last_visible_entry]);
            }
        
            temp++;
        
            if(strncmp(temp, "(DIR)", 5) == 0)
            {
                for(int i = 0; i < 5; i++)
                {
                    temp++;
                }
                
                path_append(curr_dir, temp);
                
                // print added for test purposes only
                // remove later            
                print("%s\n", curr_dir);
                
                writeChar(lcd, LCD_ON_CUR_ON_NO_BLK);
        
                index_dir_entries(curr_dir);
        
                lcd_cur_line = 0;    
                last_visible_entry = 3;
        
                update_lcd();
            }
            //add else to indicate a file was chosen
        }    
    }
    

    If you intend to assist, follow along, or use this code, here is the current pin settings for the Prop BOE
    // Prop BOE pin settings
    #define LCD_SCR_UP 2 // Pushbutton - scroll up display
    #define LCD_SCR_DWN 3 // Pushbutton - scroll down display
    #define LCD_SELECT 4 // Pushbutton - select file or directory
    #define LCD_CONFIRM 5 // Pushbutton - confirm file or directory selection, and go 
    #define LCD_PARENT 6 // Pushbutton - if not root, go to parent directory
    #define LCD_RESET 7 // Pushbutton - go to root diectory
    #define LED_YELLOW 8 // LED - currently for testing, ultimately when file is opend for reading
    #define LED_RED 9 // LED - error indicator
    #define SD_DO 22 // SD data out
    #define SD_CLK 23 // SD clock
    #define SD_DI 24 // SD data in
    #define SD_CS 25 // SD chip select
    #define LCD_RX 12 // LCD serial RX
    
  • idbruceidbruce Posts: 6,197
    edited 2015-05-08 06:13
    In reference to my previous post, I have altered clear_entry_str(char *entry) as follows:
    void clear_entry_str(char *entry)
    {
        memset(&entry, 0, strlen(entry));
    }
    
  • Heater.Heater. Posts: 21,230
    edited 2015-05-08 06:29
    You have a bug there, no & is wanted in front of the entry parameter to memset.

    If you want to clear a string in a char array all you have to do is set the first byte to zero.
    *entry = 0;
    
    or
    entry[ 0 ] = 0;
    

    That is basically putting a zero length string into the array. The rest of the bytes don't natter.

    That saves some code space and perhaps does not even warrant being in a function of it's own. A macro world do.

    If you actually want to clear the bytes of an array I would like to see the use of
    memset(entry, 0, ENTRY_SIZE);
    
    This is guaranteed never to overrun the size of entry. I know strlen() should never be to long but things go wrong...
Sign In or Register to comment.