Shop OBEX P1 Docs P2 Docs Learn Events
Concurrent SD card access — Parallax Forums

Concurrent SD card access

Has anyone been able to access the SD card via two separate cogs at the same time? For example both cogs read the same file, but at different positions in the file?

Thanks,
Terry

Comments

  • evanhevanh Posts: 15,187

    Bound to be a fail. At the very least the SD card will get confused. What should work, though, is operating two separate cards at once.

  • evanhevanh Posts: 15,187

    PS: That's the job of an OS. To deal with multiple tasks wanting access to multiples files from the one shared filesystem.

  • ke4pjwke4pjw Posts: 1,079

    I think I have a workaround. It just requires a little book keeping in my specific use case. If the SD card objects supported file handles, then it could do the book keeping. A full blowen OS doesn't need to be written to do this. I have locks that manage multiple cogs accessing the Wiznet chip. No OS needed.

  • RaymanRayman Posts: 13,860

    FAT32 has a seek function:
    fileSeek(position)

    That might let you jump around inside the same file...

  • roglohrogloh Posts: 5,158
    edited 2024-03-21 00:13

    If the SD card application is read only and you take/give a lock between sector accesses from different COGs you should be able to share reads from two or more COGs. However as soon as you introduce writes you'd then need to be mindful about filesystem structures and exactly where your lock/unlock points are so you don't clobber the files with two writes to the same sector or otherwise read stale data etc.

  • ke4pjwke4pjw Posts: 1,079

    @Rayman said:
    FAT32 has a seek function:
    fileSeek(position)

    That might let you jump around inside the same file...

    This is exactly what I am going to have to do, though I didn't think of just keeping the file open. In this specific case, that works.

    I wonder if a wrapper can be written to do file handles. The wrapper could keep track of files that are "opened", the mode they were opened in, their file position, and prevent a single file from being opened for writing by two "processes". The wrapper would have to open and close the files upon each access, have the bookkeeping in hubram, and use locks to ensure only one COG has access. Access times might not be great. This would lend itself to having a webserver that has more than 1 available port and multiple threads in separate cogs.

    At the moment, this is all a thought exercise. I think it might be useful.

  • ke4pjwke4pjw Posts: 1,079

    OK, I think a wrapper is doable. And the great thing about doing it as a wrapper, you can easily swap out the engine if another one comes along.

  • evanhevanh Posts: 15,187

    Of course you can certainly code such functionality that doesn't yet exist. Yes, the locks are a minimum.

  • pik33pik33 Posts: 2,350

    2 cogs + 1 SD card = trouble

    A driver for the card is needed, something like PSRAM driver, so one cog controls the SD and responds to requests from other cogs.

  • AJLAJL Posts: 515
    edited 2024-03-22 08:59

    Depending on the processing load for those cogs, you could run one cog with a sequence of:
    process own cog SD request
    process own cog web server code
    process other cog SD request

    It would probably be a bit slower than using 3 cogs, but if you are short on cogs…

    Edit: I suppose you could also have one cog running the driver and the other serving both web sessions if there’s enough cog ram in the web server cog to hold the pointers to the buffers for both sessions.

  • ke4pjwke4pjw Posts: 1,079
    edited 2024-03-25 05:09

    I started on a filesystem wrapper that provides filehandles and things are progressing. I think this might be useful for others.
    Quick and dirty test of open.

  • RossHRossH Posts: 5,345

    @ke4pjw said:
    Has anyone been able to access the SD card via two separate cogs at the same time? For example both cogs read the same file, but at different positions in the file?

    Thanks,
    Terry

    Probably not what you are after, but Catalina can do this - i.e. you can start C functions in different cogs and each cog can then open and read from a file - either the same file or a different file.

    Here is an example I just compiled and ran (it's a modified version of the multiple_cogs.c demo, and uses the utility functions from the demos\multicog folder).

    This version only has one cog accessing the file at any one time, but if multiple cogs may do so, just use a lock to prevent contention (you could use the same lock as is used to prevent contention on the serial port).

    /***************************************************************************\
     *                                                                           *
     *                         Multiple Cogs SD Access Demo                      *
     *                                                                           *
     * This program shows how to use the cog function coginit_C to start         *
     * many cogs running C functions that can each read files from the SD        *
     *                                                                           *
     * Compile with a command like:                                              *
     *   catalina multiple_SD.c cogutil.c -p2 -lcx -lm                           *
     *                                                                           *
     \***************************************************************************/
    
    /*
     * include the definitions of some useful multi-cog utility functions:
     */
    #include "cogutil.h"
    #include <stdio.h>
    
    #define MAX_LINE_LENGTH 128
    
    // need a lot of stack space to use stdio!
    #define MY_STACK_SIZE (STACK_SIZE + 500)
    
    /*
     * define some global variables that all cogs will share:
     */
    static int read;
    static int lock;
    
    /*
     * read_function : C function that can be executed in a cog.
     *                (the only requirement for such a function is that it 
     *                be a void function that requires no parameters - to 
     *                share data with it, use common variables)
     */
    void read_function(void) {
       int me = _cogid();
       char name[] = "fileX.dat";
       FILE *file;
       char line[MAX_LINE_LENGTH];
    
       cogsafe_print_int(lock, "Cog %d started!\n", me);
       name[4]=0x30+me;
    
       file = fopen(name, "r");
    
       if (!file) {
          cogsafe_print_int(lock, " ... cog %d failed to open file\n", me);
       }
       while (1) {
          if (read == me) {
             if (fgets(line, MAX_LINE_LENGTH, file) != NULL) {
                cogsafe_print(lock, line);
             }
             read = -1;
          }
       }
    }
    
    /*
     * main : start up to 7 cogs, then signal each one in turn
     */
    int main(void) {
       int i = 0;
       int cog = 0;
       unsigned long stacks[MY_STACK_SIZE * 7];
    
       // assign a lock to be used to avoid plugin contention
       lock = _locknew();
    
       // start instances of read_function until there are no cogs left
       do {
          cog = _coginit_C(&read_function, &stacks[MY_STACK_SIZE*(++i)]);
       } while (cog >= 0);
    
       // now loop forever, signalling each cog in turn
       while (1) {
          for (cog = 0; cog < 8; cog++) {
             read = cog;
             // slow things down a bit so we can see the messages
             wait(200); 
          }
       }
    
       return 0;
    }
    
  • ke4pjwke4pjw Posts: 1,079

    Attached is my wrapper for concurrent SD card access. When you open a file, it returns a filehandle. All operations will need to provide that file handle to read and write to the file. The methods in this object map to Nyamekye's FAT32 object, but it could be modified to support other filesystem objects.

    ToDos:

    • Directories are currently not supported. I may need to restructure the array for filenames or create a new structure for paths. This may further affect performance.
    • Error checking and return values are not great. I just don't know the best way to code this. Making it deal with aborts the bubble up out of Nyamekye's object is cooking my noodle.
    • There are many missing methods that are supported by Nyamekye's object.
    • Write methods have not been tested. Use at your own risk.
    • I had an SD card get scrambled using this wrapper. Seems to be SD card specific, as others have not had the issue.

    I will develop this further as needed. Currently it is used in my April Fools gag, and seems to work OK. Performance is not great, but it does work.

    --Terry

Sign In or Register to comment.