Shop OBEX P1 Docs P2 Docs Learn Events
Speed of Reading File from SD card is low - Page 2 — Parallax Forums

Speed of Reading File from SD card is low

2

Comments

  • evanhevanh Posts: 15,662
    edited 2023-08-24 09:04

    It's very slow, but yep. You should be seeing it working at slower sysclock too. If not then I suspect you still need a newer build of Flexspin. I'd like to see results from you testing at slower sysclock first.

    PS: It is a little temperamental. I sometimes need to power down to recover if I've got the EEPROM confused from messing with the timings.

  • chintan_joshichintan_joshi Posts: 135
    edited 2023-08-25 05:30

    @evanh said:
    It's very slow, but yep. You should be seeing it working at slower sysclock too. If not then I suspect you still need a newer build of Flexspin. I'd like to see results from you testing at slower sysclock first.

    PS: It is a little temperamental. I sometimes need to power down to recover if I've got the EEPROM confused from messing with the timings.

    I have tried with Low frequency starting from 150 MHZ without rebuilding of Flexspin, and mount working fine.
    I have also tried to increase frequency to 340 MHZ, at higher frequency its got stuck and needs to restart the board and after several try able to get the read timings. At 340 MHZ, read time to 19200 bytes is 13 ms. that's 3 times than the SD card read time.

    So Flash option seems not useful for Faster data read. Are you getting same read speed for Flash at Higher frequency?

    With 150 MHZ i have written 19200 bytes to file, not able to write data with 340 MHZ.
    And after that with 340 MHZ i have read the 19200 bytes. And got this timing.

  • evanhevanh Posts: 15,662
    edited 2023-08-25 07:24

    Read speed is peaking at nearly 1800 kB/s with sysclock/4. Not bad but yeah, SD is doing better even though it's on a larger divider because of that resistor slowing the peak SD clock rate.

    The EEPROM routines appear to use extra software layers and also the lowest level is in Spin/Pasm which is repeatedly Fcached for each SPI component (CA/read/write) within an operation. That sets up a thrashing effect of copying the SPI routines into CogRAM over and over just to read a byte. I'm surprised it goes as fast as it does.

    Could be refactored for much less bloat.

  • evanhevanh Posts: 15,662
    edited 2023-08-25 11:36

    @chintan_joshi said:
    I have tried with Low frequency starting from 150 MHZ without rebuilding of Flexspin, and mount working fine.
    I have also tried to increase frequency to 340 MHZ, at higher frequency its got stuck and needs to restart the board and after several try able to get the read timings.

    Here's what I got up to last night. The critical improvement is adding the P_INVERT_B bit to the mode line in line 35. And to make that useable at all frequencies requires a fixed clock divider of sysclock/4. This is set with the wxpin #2,#spi_ck at line 152.

    Everything else is just me doing coding preferences.

    PS: File is located include/filesys/littlefs/SpiFlash.spin2

  • evanhevanh Posts: 15,662
    edited 2023-08-25 12:08

    The reason why sysclock/4 (sysclock/5 too) is important for P_INVERT_B is because it takes 4 sysclock ticks, after its smartB clocking edge, for the TX smartpin to present its data bit to its pin. Combined with a SPI cmode 3 setup (first data bit is actually presented early so the first clock edge isn't needed for it) we use the leading falling clock edge, via the P_INVERT_B, for clocking out the second TX data bit.

    The sysclock/4 divider conveniently provides the right lag for the data bit transition to align with the ideal following (second) falling clock edge so that the second data bit is stable at the slave device on the 2nd rising clock edge, or 6 sysclocks later.

  • evanhevanh Posts: 15,662
    edited 2023-08-25 12:28

    The reason why it takes 4 sysclock ticks for the TX smartpin to present its data bit is because of:
    - One for synchronisation
    - One for input staging from SPI clock pin to smartB input
    - One for smartpin reaction
    - One for output staging from smartpin output to TX data pin

    The two staging registers is one of the sacrifices that had to be made for Prop2 clock speed during the design. It is even worse for the Cogs, they have another four staging registers, minus the smartpin, on top of this.

  • evanhevanh Posts: 15,662

    The RX smartpin doesn't have any internal clock phase lag to worry about since the SPI clock and SPI data are both inputs following identical parallel paths. The only worry here is how quick is the slave device.

  • chintan_joshichintan_joshi Posts: 135
    edited 2023-08-28 13:07

    Hello @evanh

    I have tried the two changes you suggested in include/filesys/littlefs/SpiFlash.spin2

    Changes

           modeTXD = P_SYNC_TX |P_INVERT_B| P_OE |((spi_ck - spi_di) & %111)<<24 ' + P_SYNC_IO + P_FILT0_AB
            'modeTXD = P_SYNC_TX + P_OE |((spi_ck - spi_di) & %111)<<24 ' + P_SYNC_IO + P_FILT0_AB
    
    PRI Spi_Init() | div
      div:= (clkfreq / 100_000_000) #> 4                   ' adjust smart pin base period to actual frequency
      ORG
                    fltl    #spi_di                        ' reset smart pins
                    fltl    #spi_do
                    wrpin   ##modeClk,#spi_ck              ' smart pin transition mode
                    wrpin   ##modeTXD,#spi_di              ' smart pin synchronous serial transmit
                    wrpin   ##modeRXD,#spi_do              ' smart pin synchronous serial receive
                    'wxpin   div,#spi_ck                    ' clock transition base period
                    wxpin   #2,#spi_ck                     ' clock transition base period (sysclock/4)
                    drvl    #spi_ck                        ' enable smart pin
      END
    

    But speed seems slow at 340 MHZ, 11 ms for 19200 bytes read, Write have same issue of stuck at higher frequency.

  • evanhevanh Posts: 15,662

    That's correct, as per previous post:

    @evanh said:
    Read speed is peaking at nearly 1800 kB/s with sysclock/4. Not bad but yeah, SD is doing better even though it's on a larger divider because of that resistor slowing the peak SD clock rate.

    The EEPROM routines appear to use extra software layers and also the lowest level is in Spin/Pasm which is repeatedly Fcached for each SPI component (CA/read/write) within an operation. That sets up a thrashing effect of copying the SPI routines into CogRAM over and over just to read a byte. I'm surprised it goes as fast as it does.

    Could be refactored for much less bloat.

  • evanhevanh Posts: 15,662

    Writes are extremely poor speed wise, but that's normal for all basic single block Flash erase and write.

    SD cards have internal performance hardware arrangement with parallel Flash block erase and writes, and maybe internal caching as well, to achieve their high write speeds.

  • @evanh said:
    Writes are extremely poor speed wise, but that's normal for all basic single block Flash erase and write.

    SD cards have internal performance hardware arrangement with parallel Flash block erase and writes, and maybe internal caching as well, to achieve their high write speeds.

    Yes, waiting for SD Add-on board. Only check read speed after receiving the boards. Hope to see some more speed than regular SD card read.

  • evanhevanh Posts: 15,662
    edited 2023-08-29 23:25

    @chintan_joshi said:
    Yes, waiting for SD Add-on board. Only check read speed after receiving the boards. Hope to see some more speed than regular SD card read.

    The inline resistor needs bridged out. Then the divider can be fixed at sysclock/4. That should peak at 10 MB/s.

  • @VonSzarvas said:
    Here's that diagram, with the resistor in question highlighted green!
    So remove that resistor, and add a solder blob (or bit of wire/etc.. ) across the resistor pads to short them out.
    Then you can run full speed SD!.
    Just don't plug the modified board into the top P2 addon header pins (P56-P63). Any other header pins, no problem!

    Hi, @VonSzarvas ,
    I have P2 microSD Add-on Board. Can you help me how can i start to read and write data into sd card using this module.

    Thanks and Regards
    Kundan Jha

  • evanhevanh Posts: 15,662
    edited 2023-09-05 13:45

    Instead of mount("/sd", _vfs_open_sdcard()) use mount("/sd", _vfs_open_sdcardx( clk, ss, di, do ))

    Eg: If the addon board is plugged into pin group P32..P39 then mount("/sd", _vfs_open_sdcardx( 37, 36, 35, 34 ))

    PS: I'm assuming you're using FlexC, same as Chintan.

  • @evanh said:
    Instead of mount("/sd", _vfs_open_sdcard()) use mount("/sd", _vfs_open_sdcardx( clk, ss, di, do ))

    Eg: If the addon board is plugged into pin group P32..P39 then mount("/sd", _vfs_open_sdcardx( 37, 36, 35, 34 ))

    PS: I'm assuming you're using FlexC, same as Chintan.

    Hi @evanh ,
    Thank you for your suggestions!!
    I am getting issue in file reading, below is my code and error screenshot.
    In add-on sd card module one DET pin is there. Do i need to connect.

    FILE* fh;
    int mnt = mount("/sd", _vfs_open_sdcardx( 43, 42, 41, 40 ));
    mount("/sd",_vfs_open_sdcard());
    perror("Read Mount:");
    unsigned char* filename = "/sd/image.dat"; //image.dat file already available in sd card

    if (fh = fopen(filename, "rb"))
    
    {
        printf("File opened successfully\n");
    }
    else
    {
        printf("Issue in File open, exiting\n");
        perror("File Open");
        return 0;
    }
    while(true){
        //Doing some tasks
    }
    

  • evanhevanh Posts: 15,662
    edited 2023-09-06 07:48

    Okay, base pin is P40:
    P40 is not used
    P41 is Card Detect mechanical switch
    P42 is DO (MISO)
    P43 is DI (MOSI)
    P44 is SS (CS)
    P45 is CLK

    Therefore correct parameters are: _vfs_open_sdcardx( 45, 44, 43, 42 )

    I guess a good idea is formalise that add-on card with something like _vfs_open_sdcardx( basepin+5, basepin+4, basepin+3, basepin+2 )

  • Kundan_jhaKundan_jha Posts: 17
    edited 2023-09-06 11:14

    @evanh said:
    Okay, base pin is P40:
    P40 is not used
    P41 is Card Detect mechanical switch
    P42 is DO (MISO)
    P43 is DI (MOSI)
    P44 is SS (CS)
    P45 is CLK

    Therefore correct parameters are: _vfs_open_sdcardx( 45, 44, 43, 42 )

    I guess a good idea is formalise that add-on card with something like _vfs_open_sdcardx( basepin+5, basepin+4, basepin+3, basepin+2 )

    @evanh said:
    Okay, base pin is P40:
    P40 is not used
    P41 is Card Detect mechanical switch
    P42 is DO (MISO)
    P43 is DI (MOSI)
    P44 is SS (CS)
    P45 is CLK

    Therefore correct parameters are: _vfs_open_sdcardx( 45, 44, 43, 42 )

    I guess a good idea is formalise that add-on card with something like _vfs_open_sdcardx( basepin+5, basepin+4, basepin+3, basepin+2 )

    Thanks @evanh
    Now its working.

  • @evanh said:

    @chintan_joshi said:
    Yes, waiting for SD Add-on board. Only check read speed after receiving the boards. Hope to see some more speed than regular SD card read.

    The inline resistor needs bridged out. Then the divider can be fixed at sysclock/4. That should peak at 10 MB/s.

    Hello @evanh Just received SD card Addon Module, i have removed the resistor as suggested and shorted both points.

    Tried to run at 340 MHZ with sysclock/4 configuration in sdmm.cc file as below

    #ifdef _smartpins_mode_eh
                    tmr = _clockfreq();
                    spm_tx |= P_INVERT_B;
                    // Performance option for "Default Speed" (Up to 50 MHz SPI clock)
                    if( tmr <= 150_000_000 )  ck_div = 0x0002_0004;  // sysclock/4
                    else if( tmr <= 200_000_000 )  ck_div = 0x0002_0005;  // sysclock/5
                    else if( tmr <= 280_000_000 )  ck_div = 0x0002_0006;  // sysclock/6
                    //else  ck_div = 0x0003_0008;  // sysclock/8
                    else  ck_div = 0x0002_0004;
    

    But its behaving same , Not accepting sysclock/4 and giving io error to read the file.
    sysclock/5 working same as default sd card module. so am i missing something to achieve more speed using addon module?

  • evanhevanh Posts: 15,662

    sysclock/5 at 340 MHz is working for you? That's probably the best it'll do then. You should be good for 8 MB/s.

  • evanhevanh Posts: 15,662
    edited 2023-09-07 09:26

    Hmmm, actually, something has changed in Flexspin. My testing is now erratic mounting. Looks like some confusion during the SD init sequence. The card type is not detecting correctly on every mount attempt. ...

    EDIT: Turns out it was a new SD card creating its own little problems.

  • @evanh said:
    sysclock/5 at 340 MHz is working for you? That's probably the best it'll do then. You should be good for 8 MB/s.

    @evanh, yes sysclock/5 was working in default sd card slot provided in P2 as well. I have also mentioned in this comment.

    https://forums.parallax.com/discussion/comment/1553052/#Comment_1553052

    But query is no performance difference i have seen in default sd card slot of P2 and Addon card without resistor. Both are giving same read speed.

  • evanhevanh Posts: 15,662
    edited 2023-09-07 05:15

    I had assumed you were using a lower sysclock in that earlier situation, since sysclock/5 at 340 MHz isn't possible without bridging out the inline resistor first.

    You should measure the peak data rate to know behaviour better. That means reading larger file sizes or doing the time measurement within the low-level code to separate filesystem overheads from SD performance.

  • @evanh said:
    I had assumed you were using a lower sysclock in that earlier situation, since sysclock/5 at 340 MHz isn't possible without bridging out the inline resistor first.

    You should measure the peak data rate to know behaviour better. That means reading larger file sizes or doing the time measurement within the low-level code to separate filesystem overheads from SD performance.

    oh yes, you are right, i was just checking again and found that earlier sysclock/5 worked with lower clock.

  • @evanh said:
    I had assumed you were using a lower sysclock in that earlier situation, since sysclock/5 at 340 MHz isn't possible without bridging out the inline resistor first.

    You should measure the peak data rate to know behaviour better. That means reading larger file sizes or doing the time measurement within the low-level code to separate filesystem overheads from SD performance.

    Let me check with original file which was 2.4 MB long.

  • chintan_joshichintan_joshi Posts: 135
    edited 2023-09-08 10:26

    i have checked with original file, its giving read speed of 2.8 to 3.6 ms for 19200 bytes. Clock frequency is 340 MHZ, in sdmm.cc file changed clock_div to ck_div = 0x0002_0005;, sysclock/5, but seems its not increasing speed.

  • evanhevanh Posts: 15,662
    edited 2023-09-08 11:48

    You do realise that's an average of 6.7 MB/s. For a small file with filesystem overheads and a peak of 8 MB/s you're actually doing pretty good.

    Like I said, a larger file will be more telling of how much is overheads verses SD performance.

  • chintan_joshichintan_joshi Posts: 135
    edited 2023-09-12 05:06

    Hello, i have developed function to read some data from sd card file, I am able to build and run this function in Flexprop 6.0.2 but when i tried to use Flexprop version 6.4.0 its not compiling.

    Below is the function.

    int stepValues[6];
    int readStepValues() {
        int maxSteps = 6; 
        mount("/sd", _vfs_open_sdcard());
        unsigned char* filename = "/sd/steps.txt";
        FILE* file;
        if (file = fopen(filename, "r"))
        {
            printf("File opened successfully\n");
        }
        else
        {
            printf("Issue in File open, exiting\n");
            perror("File Open");
            _cogatn(1 << 0);
            return 0;
        }
    
    
        int count = 0;
        char line[100];  // Adjust the buffer size as needed
        while (count < maxSteps && fgets(line, sizeof(line), file) != NULL) {
            if (line[0] != '#' && sscanf(line, "%d", &stepValues[count]) == 1)  // This line generating this error and warning
           {
                count++;
            }
        }
    
        fclose(file);
        umount("/sd");
        return count;
    
    }
    

    And compile time error i am getting in Flexprop 6.4.0 is below

    C:/Users/admin/Desktop/Flexprop/flexprop-6.4.0/flexprop/include/libc/wchar/mbrtowc_utf.c:30: warning: incompatible pointer types in assignment: expected pointer to _struct___Mbstate but got pointer to any
    error: No implementation for `mbrtowc' found in `libc/wchar/mbrtowc_utf.c'
    
  • evanhevanh Posts: 15,662
    edited 2023-09-12 08:49

    There's no includes. EDIT: Okay, stdio.h seems to be enough. Yep, getting same error.

  • @evanh said:
    There's no includes. EDIT: Okay, stdio.h seems to be enough. Yep, getting same error.

    I have used below header files , some of them are user defined. Seems its not an issue of header file.

    #include <stdio.h>
    #include <propeller2.h>
    #include "simpletools.h"
    #include <smartpins.h>
    #include "media_Header.h" // Header file for all the structure defination for Media sleeves movement
    #include "Pin_define.h"
    #include "filldata.h"
    
    #include <stdint.h>
    #include <stdlib.h>
    #include <math.h>
    #include <sys/time.h>
    #include "TSR.h"
    
  • The mbrtowc bug is fixed in github source code now. In the meantime, you could work around it by using strtol() in place of sscanf (this would also make the code a bit smaller and faster, scanf is quite slow).

Sign In or Register to comment.