P2 Spin2 Library for SD card Fat32 file system

travelerdawg

Primary goal of this discussion - find a Spin2 library that allows the P2 to write a txt file to the SD card of the P2 Eval board.

Ultimate end goal- utilize the built in SD card reader of the P2 Eval board to write 4 txt files while simultaneously recording 4 input sources from sensor data. The idea is the P2 would collect data in buffers from the 4 sources and write to the SD card when available. The P2 would act as a data collection device and later act as a USB host to present the contents of the SD card to a Windows PC. The Windows PC would then be able to copy these txt files onto the hard drive of the PC.

This is for an embedded project where the SD card would not be available to remove from the board once installed. We were able to do this with 2 data sources and writing to 2 open files utilizing the P1 and an ALFAT. The ALFAT is no longer available and our data sources have doubled. We would like to simplify our design by only using the P2 if possible. The ALFAT was able to create the file structure and act as a USB host in our previous design.

evanh

Flexspin/Flexprop offers both FAT32 on SD cards and also serial file transfer via loadp2 as file-server - https://www.parallax.com/propeller-2/programming-tools/

Dunno if they're usable from Spin though. Maybe just C and Basic.

travelerdawg

@evanh Thanks! I was really hoping for something for SPIN2 but I will chase this Flexprop rabbit as far as I can get it! I've never used Flexprop, so this will be a new experience.

msrobots

sure they are, you just need to include a C or Basic object in your Spin file to do so.

Mike

travelerdawg

Need a little more guidance on this if possible. Not sure where to find the loadp2 as a file server. It doesn't look like the Prop Tool can use C or anything other than SPIN1, SPIN2 and PASM.

Has anyone used the P2 to access the SD card on the Eval board?

ersmith

Unfortunately there aren't many good examples yet (contributions are welcome!) but FlexProp BASIC does have full support for reading/writing files on the SD card and on the host PC's file system. The FlexBASIC manual (accessible via Help > BASIC Language from FlexProp, or you can read the basic.pdf file) describes the functions, which will be pretty familiar if you've used any flavor of MS BASIC. Look particularly at the OPEN, CLOSE, PRINT, and INPUT commands for doing reading and writing files, and the CHDIR, DIR$, and MKDIR commands for manipulating directories. Everything starts with the MOUNT command. So for example to print "hello, world!" into a file called "hello.txt" on the SD card, you would do:

'
' Demo for writing to a file
' Defaults to the SD card, but you can change the commented lines to make it work
' on the host PC's file system instead
'

' for SD card
mount "/dir", _vfs_open_sdcard()

' for Host
'mount "/dir", _vfs_open_host()

' open the file for writing
open "/dir/hello.txt" for output as #3
print #3, "Hello world!"
close #3

To do all this from Spin2 is possible in FlexProp, but slightly trickier. The easiest way is probably to do the file I/O in BASIC, and then include that BASIC file as an object in Spin2. This will only work in FlexProp, not PropTool.

travelerdawg

@ersmith you have knowledge far and away beyond what I am capable of understanding when it comes to Flexprop, BASIC, and just...programming. Where is the basic.pdf? Because, I have a hard time finding any info on the main Parallax page associated with getting started guides. There are so many IDEs that can work for this hardware yet there are really no good tutorials on how to get started with those IDEs. I mean, "Hello World" is fine and all, but that is about all there is available. (I have had success a couple of the "Quick Bytes") I can program an MSP430 (pain in my butt BTW) but working with the Propeller 2 is a mountain of information that makes no sense. Honestly, if you have ever had to deal with the MSP430, that is saying something...

There are literally 4 lines of code in your example (minus the comments). I understand the "mount" command but have no idea what library it originates from. I am (making a LOT OF assumptions) assuming you are somehow incorporating a library from Help>BASIC Language in Flexprop? This is one of my biggest complaints/misunderstandings with "Getting Started" with propeller. There aren't any clear cut instructions on how to operate in any particular IDE (Propeller Tool or FlexProp) vs the hardware.

In your four line program, you have zero pins specified for the SD card. Are those pins specified in the library? Is this library specific for the P2 Eval board or some other piece of hardware? How would I know? What is #3? I am assuming, based on my previous programming knowledge, that #3 is somehow a file handle? How was #3 selected as the file handle? Is this number arbitrary and assigned by the programmer or is there a select number of file handles? I am assuming the "open" command is somehow a FileHandle command? I am familiar with FileHandles from other IDEs but a NEW USER might have zero understanding of this. The 'print' is most likely similar to the 'print' command used to print to console but instead of printing to console this command prints to a file, but how does the program know to 'print' to a file instead of 'print' to the console?

I really hope that @ersmith, the community, and the team with Parallax sees this comment in this discussion. I am trying my best to move forward with the Propeller. I want to see this platform succeed because I firmly believe it is a better platform than any other microcontroller out there. However, the ENTRY to BEGIN seems terribly high! My education includes a Bachelors in Computer Engineering and I am looking at the forums, comments, and P2 documentation and I'm like, "Why is this not easier for a New User?"

I look at the documentation and it is like reading one of my old textbooks. I mean no disrespect to users or those currently working on documentation. I don't know where you are in that process. All I know is where I am in the process of attempting to learn how to program for the Prop2 and I feel frustrated.

I am hoping someone will point something out I am missing and that I will feel like an idiot for not catching something and then I will need to apologize for not getting it.

evanh

FlexProp is not a Parallax product. You'll want to be installing from Eric's repo - https://github.com/totalspectrum/flexprop/releases

EDIT: Oh, that's a lot for you to learn I guess. I've not used Basic (nor the FlexProp IDE) on the Propellers myself. I usually end up putting my foot in it when answering questions without testing myself. The guys in the Basic forum seem pretty active - https://forums.parallax.com/categories/basic

Electrodude

The P2 is still in a sort of early-adopter pre-release limbo where it's in production and you can buy the part and use it, but official documentation and support isn't exactly where it should be. Information is scattered everywhere, most of it in several Google docs but a significant amount buried in various forum threads. There exists a small subset of documents together containing all you'll need to know if you look hard enough, but AFAIK there's no authoritative list of which ones these are because the official docs aren't yet done.

Parallax says their official documentation should be done soon and that they'll officially release the P2 in August.

JRoark

@travelerdawg said:
@ersmith you have knowledge far and away beyond what I am capable of understanding when it comes to Flexprop, BASIC, and just...programming. Where is the basic.pdf? Because, I have a hard time finding any info on the main Parallax page associated with getting started guides. There are so many IDEs that can work for this hardware yet there are really no good tutorials on how to get started with those IDEs. I mean, "Hello World" is fine and all, but that is about all there is available. (I have had success a couple of the "Quick Bytes") I can program an MSP430 (pain in my butt BTW) but working with the Propeller 2 is a mountain of information that makes no sense. Honestly, if you have ever had to deal with the MSP430, that is saying something...

Heheh. Yep, there is wee bit of a curve here. We've all been there. The doc you are looking for is in /docs/basic.pdf

I understand the "mount" command but have no idea what library it originates from.

It's part of the BASIC language, like PRINT, INPUT, etc. So there is no need to declare anything. "It just works".

In your four line program, you have zero pins specified for the SD card. Are those pins specified in the library? Is this library specific for the P2 Eval board or some other piece of hardware? How would I know?

There is no need to specify the pins if you are using a stock P2-EVAL. Again, these are defaults and with a P2-EVAL board it "just works". You only need to specify pins if you are not using the default SD slot by using MOUNT with the **_vfs_open_sdcardx(clk, sel, di, do) ** argument. (Page 64 of manual)

What is #3? I am assuming, based on my previous programming knowledge, that #3 is somehow a file handle? How was #3 selected as the file handle? Is this number arbitrary and assigned by the programmer or is there a select number of file handles?

Correct as to it being a file handle. Correct that it is assigned by the programmer. Valid file handles range from 0-7, with 0 and 1 being used internally for the serial tx/rx to and from the host PC. Handle #3 is just an available unused handle. (See also Page 37, under CLOSE)

I am assuming the "open" command is somehow a FileHandle command? I am familiar with FileHandles from other IDEs but a NEW USER might have zero understanding of this. The 'print' is most likely similar to the 'print' command used to print to console but instead of printing to console this command prints to a file, but how does the program know to 'print' to a file instead of 'print' to the console?

BASICs tend to support PRINT both to the console (ie, PRINT "Hi there") and to a file (ie, "PRINT #3 "Hi there"). If you specify a file handle, PRINT simply directs its output to that handle.

I really hope that @ersmith, the community, and the team with Parallax sees this comment in this discussion. I am trying my best to move forward with the Propeller. I want to see this platform succeed because I firmly believe it is a better platform than any other microcontroller out there. However, the ENTRY to BEGIN seems terribly high! My education includes a Bachelors in Computer Engineering and I am looking at the forums, comments, and P2 documentation and I'm like, "Why is this not easier for a New User?"

The source of your frustration is probably because you haven't yet had the docs in your hand. The docs are terse, but they are there! See the /docs/basic.pdf file in the FlexProp distribution. It's just short of 100 pages currently, and covers everything just discussed.

I look at the documentation and it is like reading one of my old textbooks. I mean no disrespect to users or those currently working on documentation. I don't know where you are in that process. All I know is where I am in the process of attempting to learn how to program for the Prop2 and I feel frustrated.
I am hoping someone will point something out I am missing and that I will feel like an idiot for not catching something and then I will need to apologize for not getting it.

No idiots (except me) on this board.
I think your frustration is temporary. Once you get your hands on the docs, it will be a lot clearer. Please understand that FlexProp is the work of @ersmith. He isn't a Parallax employee and none of his compiler suite has been "officially" sanctioned by Parallax. He was just this guy who went "Man, we need this", and did it. The compiler suite includes SPIN, SPIN2, PASM, PASM2, BASIC, and C. That was a herculean effort for one guy, and yes, the docs are not "mature". But they are there.

Parting thought: use the IDE provided with FlexProp... at least initially. It's dirt-simple and it works pretty well. Dont try (yet) to install something like VSCode or you'll be fighting not just a new language implementation, but also a new debugger for that language. Take little bites when starting out.

If you have questions or need examples, post here and we'll try to help you out.

ADDIT: Doc attached.

travelerdawg

@JRoark bless you and thank you kind person!
You broke my inquiries down and answered them consecutively, got me tearing up over here. AND you attached the document in question! I will take your advice and stick with FlexProp before attempting VSCode. It was temping to jump in because I am more used to writing C# in Visual Studio.

This community is awesome! I am looking forward to continuing my journey with y'all.

ersmith

@JRoark : thank you for your detailed answer!

@travelerdawg : I fear the discussion is starting to stray away from "Spin2 library for SD Card FAT 32" (the title of the thread). If you would like to try doing the programming in C or BASIC, perhaps it would be good to open a discussion in one of those forums... FlexProp can certainly handle this in either language.

Cluso99

@travelerdawg
I use VSC to edit my P1 & P2 code and then FlexProp via the command line (batch file) to compile/download/run.
I am use VSC for my work related python programming so it makes sense to use an editor that is familiar. There is a VSC P2 extension - Spin2/Pasm2 Syntax... by Iron Sheep Productions. There is a Quickbytes for this as well as a thread for VSC in the P2 forums.
BTW Not sure about BASIC use though.

The P2 has a standard set of pins for SD SPI so it can boot from SD directly.

Ariba

I tried to use the SD card driver of flexProp with Spin2. Finally found a way to make it work, but I don't know if it's the intended way.
To read the file "test.txt" on SD card and show the first 511 chars in the Terminal:

VAR
  long f1
  byte bstr[512]

OBJ
   c : "libc.a"               ' C standard library

PUB main() : err | i,ch
  err := _mount(@"/sd", c._vfs_open_sdcard())
  debug(sdec(err))
  ifnot err
    f1 := c.fopen(@"/sd/test.txt",@"rb")
    debug(sdec(f1))
    ifnot f1
      debug("Error at open")
    else
      c.fread(@bstr, 511, 1, f1)
      debug(zstr(@bstr))
    c.fclose(f1)

Strange is that c.mount() gives an error, but _mount() works. There are a number of warnings because of incompatible types.
I can't get it to work with vfs_open_host() this seems broken in the newest flexProp version (also the shell.c example can't open the host).

For writing a file you would need to open the file for write (@"wb") and use c.fwrite() instead of fread. I have not tried that yet, I need to find first a lesser importand SD card ;-).

Andy

ersmith

@Ariba : The need to use _mount() rather than c.mount() is a quirk of how mount is implemented in the C library, and I'll try to fix that.

I'm more concerned that you are having trouble with the host file system. Can you give us some details of your setup? I tried the shell.c example with FlexProp 5.9.11 on Windows 10, using a WiFi connection, and it seemed to work fine. Ive been using FlexProp 5.9.11 (and 5.9.12-beta) on Linux with a serial connection and had no trouble accessing /host there either.

Wuerfel_21

Yep, that's how you can use it in Spin - there's some functionality missing/inaccessible, but open/read/write/close works great. This may be an area to improve...

travelerdawg

@ersmith I see what you mean about the title of this Discussion. I will try to remain on that topic and look for a Discussion in the FlexBasic section for help with that. I tried your example code using FlexProp, I got an error message I will put in the other Discussion thread.

@Ariba is the "libc.a" a C library? Are you using that as SPIN2 using the PropTool, is that right? I didn't know that could be done. If possible, please let me know of your success with this. Is the libc.a included with the PropTool download as one of the libraries or does it need to be added to a certain directory?

ersmith

@travelerdawg : "libc.a" is indeed a C library, and as such can only be used with Spin2 inside FlexProp. PropTool cannot handle C.

travelerdawg

As a new user, I have run into a BUNCH of minor issues that I am sure many of you just take as common knowledge.

1 Biggest issue I have had, I come from an environment of development with structured IDEs. Most of these IDEs will name the file for you or even create the entire file structure you need. Something I have been riding the struggle bus with is just getting simple code to compile. I keep getting error messages. As it turns out, the file extension was incorrect for the code I was using, Flexprop was trying to compile for me, I just wasn't giving it good information. I didn't realize the Flexprop IDE (and Propeller Tool) default to .spin for file extensions. This means I have been banging my head for the past couple days trying to figure out if there was a configuration issue that I overlooked in the documentation. As far as configuration documentation, I have read but don't really see anything to configure unless I am trying to do something out of the ordinary.

2 I have never used BASIC, have a bit of familiarity with SPIN1, never used Flexprop, and am just not sure where to start? I have another Discussion where I ask this question but whew, it is a struggle to get started with so many options, no official documentation, etc...

3 I am very appreciative for the forum and the answers I get here. I was able to get @ersmith code to work in Flexprop and create a file using BASIC! I realized while working with the code @Ariba posted is reading a file that already exists, smh, I totally missed this at first. That's what I get for not taking particular time to read his post and read through the code.

So, @Ariba and @ersmith with the code that Ariba posted, I am able to compile and I guess it is running on the P2. However, the output to the 'debug' console isn't being displayed (this is probably where I need to configure something). The Flexprop ANSI terminal is opening but nothing is being displayed except the way to exit the terminal. I have a file on the SD card to read. Is there a way to use the 'debug' with Flexprop so I can use this SPIN2 code with the C library in Flexprop? I am going to look in the forums and see if there is a thread about using debug in Flexprop.

ersmith

@travelerdawg said:
2 I have never used BASIC, have a bit of familiarity with SPIN1, never used Flexprop, and am just not sure where to start? I have another Discussion where I ask this question but whew, it is a struggle to get started with so many options, no official documentation, etc...

If you're more familiar with C than with BASIC, feel free to use that! Or, for that matter, with FlexProp you can use Spin1 on P2 (you just have to use P2 assembly instead of P1 assembly in any DAT sections).

There are some examples that come with FlexProp (in the flexprop\samples\ folder) which may be of some help. And there are the manuals... the C and Spin manuals are quite incomplete, as they assume familiarity with the respective languages already, but the BASIC manual tries to be fairly self-contained.

So, @Ariba and @ersmith with the code that Ariba posted, I am able to compile and I guess it is running on the P2. However, the output to the 'debug' console isn't being displayed (this is probably where I need to configure something). The Flexprop ANSI terminal is opening but nothing is being displayed except the way to exit the terminal. I have a file on the SD card to read. Is there a way to use the 'debug' with Flexprop so I can use this SPIN2 code with the C library in Flexprop?

The Spin2() DEBUG() statement is just that, for debugging, and so it's only enabled for "debug" compiles. That's true in both FlexProp and PropTool. I'm not sure how you enable a debug build in PropTool, but in FlexProp you go to the Options menu and select either "Print debug" or "BRK debug (P2 only)". If you're compiling for P2 then BRK debug is probably the best choice (it allows DEBUG statements inside of assembly).

Regards,
Eric

Ariba

@ersmith said:
@Ariba : The need to use _mount() rather than c.mount() is a quirk of how mount is implemented in the C library, and I'll try to fix that.

I'm more concerned that you are having trouble with the host file system. Can you give us some details of your setup? I tried the shell.c example with FlexProp 5.9.11 on Windows 10, using a WiFi connection, and it seemed to work fine. Ive been using FlexProp 5.9.11 (and 5.9.12-beta) on Linux with a serial connection and had no trouble accessing /host there either.

I tried again the shell.c and got it to work. I had to lower the baudrate to 230400 or 115200 to make it work. Also accessing a file from the host with the file commands works, if I use C. But it does not work with Spin2 and the C library. It hangs at the file open routine, and there is a warning that fopen() gets redefined with incompatible types.

@travelerdawg : I think Eric has answered all your questions (he knows FlexProp much better than me).

Andy

travelerdawg

VAR
  long f1
  long f2
  long f3
  long f4
  byte bstr[512]

OBJ
   c : "libc.a"               ' C standard library

PUB main() : err | i,ch
  err := _mount(@"/sd", c._vfs_open_sdcard())
  debug(sdec(err))
  ifnot err
    f1 := c.fopen(@"/sd/test1.txt",@"wb")
    f2 := c.fopen(@"/sd/test2.txt",@"wb")
    f3 := c.fopen(@"/sd/test3.txt",@"wb")
    f4 := c.fopen(@"/sd/test4.txt",@"wb")
    debug(sdec(f1))
    debug(sdec(f2))
    debug(sdec(f3))
    debug(sdec(f4))
    ifnot f1
      debug("Error at open")
    else
      c.fwrite(@bstr, 511, 1, f1)
      debug(zstr(@bstr))
    ifnot f2
      debug("Error at open")
    else
      c.fwrite(@bstr, 511, 1, f2)
      debug(zstr(@bstr))
    ifnot f3
      debug("Error at open")
    else
      c.fwrite(@bstr, 511, 1, f3)
      debug(zstr(@bstr))
    ifnot f4
      debug("Error at open")
    else
      c.fwrite(@bstr, 511, 1, f4)
      debug(zstr(@bstr))
    c.fclose(f1)
    c.fclose(f2)
    c.fclose(f3)
    c.fclose(f4)

@Ariba basically copy and paste. I was able to get the P2 to create files, well, only two files. I suffered this in the BASIC code as well. I was only able to create two files at a time. I was able to get 'debug' to work in FlexProp so I could see the output and I am getting the "Error at open" message for f3 and f4. I am trying to go down multiple forum posts and figure out how this all works. Maybe I need to dedicate a specific cog in order to handle the SD card handling?

ersmith

@travelerdawg : it's probably running out of memory (the file system code has to allocate some memory for each file). You could try adding:

CON
  HEAPSIZE = 32768

at the top of your main Spin file to give more memory for the run time heap.

Wuerfel_21

Note that you can print errno when a libc operation fails to figure out why. With current git builds you even get strerror in libc.a for user-readable errors.

travelerdawg

@ersmith this absolutely solved my issue with opening four files at a time! I was wondering if it was a memory issue but wasn't sure about how to fix it. Thanks!
@Ariba I am learning bit by bit and am very appreciative for your example code. I am expanding it, iterating in small pieces, to get my desired results. When I figured out how to get the 'debug' print outs working in FlexProp, I knew the program wasn't getting hung on anything as the debug messages were printing out. I have a LOT more left to learn. Here is the current working code, my next mission is to actually write 512 different bytes each of something to the files successfully.

CON
  HEAPSIZE = 32768
VAR
  long f1
  long f2
  long f3
  long f4
  byte bstr1[512]
  byte bstr2[512]
  byte bstr3[512]
  byte bstr4[512]

OBJ
   c : "libc.a"               ' C standard library

PUB main() : err | i,ch
  err := _mount(@"/sd", c._vfs_open_sdcard())
  debug(sdec(err))
  ifnot err
    f1 := c.fopen(@"/sd/test1.txt",@"wb")
    f2 := c.fopen(@"/sd/test2.txt",@"wb")   
    f3 := c.fopen(@"/sd/test3.txt",@"wb")
    f4 := c.fopen(@"/sd/test4.txt",@"wb")
    debug(sdec(f1))
    debug(sdec(f2))    
    debug(sdec(f3))
    debug(sdec(f4))
    ifnot f1
      debug("Error at open")
    else
      c.fwrite(@bstr1, 511, 1, f1)
      debug(zstr(@bstr))
    ifnot f2
      debug("Error at open")
    else
      c.fwrite(@bstr2, 511, 1, f2)
      debug(zstr(@bstr))
    ifnot f3
      debug("Error at open")
    else
      c.fwrite(@bstr3, 511, 1, f3)
      debug(zstr(@bstr))
    ifnot f4
      debug("Error at open")
    else
      c.fwrite(@bstr4, 511, 1, f4)
      debug(zstr(@bstr))    
    c.fclose(f1)
    c.fclose(f2)
    c.fclose(f3)
    c.fclose(f4)