P2 spin2/C - different behavior when running on new cog

shannon1949

I have a large (220Kb) spin2 application running on the P2 edge module that uses the SD card for data storage using flexspin/flexprop 4.1.0 and the C lib for SD card access. The object’s functions for mounting the SD card and opening/writing files work flawlessly when called from the main object and from child objects running on the main cog.

But the c.fopen() function in my object fails (and returns a file handle of zero) when called from any object running on a new/different cog that I fire up with cogspin(). The object is below; see open_append(). Debug statements show that the filename pointer and the filename string look OK. All other objects running in the new cog work fine.

I've increased allocation of stack space in cogspin() and also tried allocating heap in spin2 (per flexspin doc) in case the C functions need it. But I am out of ideas!

OBJ
   c : "libc"               ' C standard library

'' --------------
'' Mount SD card
'' --------------    
PUB mount_card() : err_mount | i, ch, tries, max_tries
  err_mount := -1
  max_tries := 3
  tries := 0
  repeat until err_mount == 0 or tries == max_tries
    err_mount := _mount(@"/sd", c._vfs_open_sdcard())
    ' debug("Mount card: ", udec_long(err_mount))
    tries++
    if tries == max_tries      
      return err_mount    
    waitms(1000)

'' --------------
'' Open file for appending; it is created on first call
'' --------------    
PUB open_append(file_name_ptr) : file_handle | tries, max_tries 

  ' debug("oa:", udec_long(file_name_ptr))

  ' debug("oa: ", zstr(file_name_ptr))

  max_tries := 2
  repeat tries from 1 to max_tries
    ''  a+b == Open for appending in binary mode.
    file_handle := c.fopen(file_name_ptr, @"a+b") 
    debug("oa:", udec_long(file_handle))
    ifnot file_handle
      debug("Error at a+b open, ", sdec_long(tries))
      if tries == max_tries
        debug("...last try failed")
        return file_handle
    else
      return file_handle

Wuerfel_21

FlexC's filesystem library is not (yet?) designed to work with multiple cogs. You should only use it in one cog (which has to be the one that did the mount call).

shannon1949

Got it. Thanks! I can make it work. I did not think of that, since all code is resident in Hub memory so (ahem) "could" have the same view of the external world including the SD card. But there must be isolating run-time code that is compiled in under cogspin(). Are there sources to help me learn more about these details? I am wondering about what run-time code is compiled into Spin2 and C programs. I think I am not knowledgeable enough on compilers/linkers, especially for multicore.

Wuerfel_21

I think the main problem that prevents it from working at all is that each cog has it's own I/O registers that are only set up during mount. But if that wasn't the case, you'd still run into problems because the code is not designed to be reentrant, so 2 cogs doing FS operations simultaneously would cause interference.

evanh

Yeah, that's probably my fault. I have a habit of leaving the clock smartpin running, in all my drivers, at the end of the transaction so that it can be pumped with extra pulses as the routine exits. Handy to ensure the device is ready for an immediate command upon next transaction. But it also means that that driver must be used by one cog only.

I haven't tried to do otherwise, therefore I wouldn't know if there is other reasons. Maybe that's something I should test out ...

evanh

Hmm, it's a little more tricky than just sneaky habits. The driver kind of needs to stay put on the cog it was initialised on. The clock pin, for example, stays driven as an output by a particular cog, irrespective of activity. This is true even when all the pins are bit-bashed.

On the good side, I've had no issue spawning the entire speed tester loop on its own cog. It starts the driver, mounts the SD card, then reads and writes multiple files.

evanh

Ha! I got it to work in the bit-bashed driver (sdmm_bashed.cc) by floating all the pins upon each deselect() and just assuming the pull-up resistors will do the job of holding the pin levels.

So I guess that proves the problems are solely in the driver code alone.

It's also another feather in the cap of using SPI cmode = 3 instead of 0. Clock idling high fits with the pull-ups.

EDIT: The tester was crashing after three loops ... Oops, I'd forgotten that it needed a huge stack for the buffer allocation. The allocations get bigger as it goes.

EDIT2: Hmm, the spec only seems to have a pull-up on CS pin on the card's side. Clock and data pins look to be devoid of pull-ups unless the board design adds them. Which is only present when the EEPROM chip is also connected.

Can't even have the Prop2 do the pull-ups, since that still requires a cog to set DIR high. Which is where the problem is - All cog output controls are OR'd together. A high on DIR is an override to all other cogs.

EDIT3: Maybe, with smartpins engaged, don't need to float anything ... With something like clock gen, DIR can be lowered and the smartpin still drives the pin. It's just in reset state.

evanh

The smartpin version (sdmm.cc) will need some work for sure. As will the SD mode driver (sdsd.cc), since it doesn't even have the select() and deselect() functions. Its design assumes a single card exclusive bus arrangement.

Is anyone all that interested in fixing this?

evanh

Well, after sleeping on it I've decided to have a careful look at the smartpins version (sdmm.cc) and discovered I wasn't doing any sneaky clocking there at all. I remember now I'd considered doing it but since the pre-existing structural code path made no use of such a feature I never went ahead.

Turns out the problem was I was setting up the tx smartpin to produce all $ff data output when not attempting to transmit anything specific. This setup was done at the end of tx low level routine and relied on tx shifter being left active. ie: DIR stayed high upon exit. I just had to change it to lowering DIR while the clock wasn't active, then raising it again later before the clock was next activated.

So, now just the 4-bit SD mode driver to sort out. I expect that's a structural change needed there.

PS: Here's the patched sdmm.cc driver, in plug-in driver form.
PPS: And an example of mounting the plug-in drivers from Spin2.

evanh

I'll be damned, it turned out to be quite clean and straight forward to also get sdsd.cc able to move around cogs too.

The sneaky clocking didn't really pose a hindrance. Its main benefits are gained during the command and response sequencing, it has no real use in having trailing clocks after a transaction is done. All I had to do was focus on lowering all the DIRs/OUTs at the end of each transaction. And for the most part, the driver already grabbed the DIRs/OUTs at the beginning of every transaction anyway. The card busy check was the only part that needed a little extra love.

PS: Obex now updated - https://obex.parallax.com/obex/sdsd-cc/

shannon1949

Thanks @evanh for the work on this. My solution to the "mounting-cog" limitation turned out to be a good fit to my needs. A cog running real time code can message the mounting-cog using COGATN()/WAITATN() [the latter in a loop], which seems to work robustly. The bonus is a cog running code in real time can quickly write a message in global memory and perform the strobe, and the mounting-cog thread can open the file on the SD, write the message, and close it --- which would be too slow for the real-time thread. In my case, the upper-level goal is to be able to generate info, warning, and error messages that are saved for (possibly post-mortem) analysis even when execution halts or goes sideways due to a run-time error.

One background reason for this is that I am playing with heap and stack sizes somewhat blindly at this point for an application that needs to run for days/weeks/+. Wondering if anyone has implemented stack canaries on the P2. Also, as a newbie, is there a way to look at the memory map from flexspin/prop? I worry that I will run out of Hub RAM for signal processing at some point.

evanh

@shannon1949 said:
... The bonus is a cog running code in real time can quickly write a message in global memory and perform the strobe, and the mounting-cog thread can open the file on the SD, write the message, and close it ...

Cool, yep, sounds right.

I've not tried to analyse any memory usage patterns. Every program I've written so far is too simple.

shannon1949

A correction to my previous post: for this to work in a loop, the protocol is COGATN()/POLLATN() with an appropriate WAITUS() in the loop. You could use WAITATN(), but if no message strobe occurs, the "mounting-cog" will hang. It also allows this cog to do other housekeeping chores.

This is not perfect. If there are two messages/strobes too close in time, the second could be lost if the strobe is not cleared in time. But it will work in most cases where messages are not posted too frequently. It would be nice to check how many clock ticks the open/write/close on the SD takes, which is the limiting factor.

ersmith

@shannon1949 said:
One background reason for this is that I am playing with heap and stack sizes somewhat blindly at this point for an application that needs to run for days/weeks/+. Wondering if anyone has implemented stack canaries on the P2. Also, as a newbie, is there a way to look at the memory map from flexspin/prop? I worry that I will run out of Hub RAM for signal processing at some point.

The listing file produced by the -l flag is quite useful for this kind of thing (or see Open listing file under the FlexProp File menu).

evanh

@shannon1949 said:
... It would be nice to check how many clock ticks the open/write/close on the SD takes, which is the limiting factor.

Don't count on that being consistent. SD cards are notorious for wobbly write timings, and even reads can be inconsistent. Some cards are more consistent than others but I wouldn't put bets on any particular brand however. Each card's firmware revision is a lottery.

PS: There's supposedly special video recording modes of operation that, I presume, are intended for smoothing the write speed. They require a bunch of allocation setup and tracking management and I think also require 1.8 Volt UHS to be engaged. It's a lot of support code and I wonder if maybe it's considered obsolete these days. UHS-II introduced a massive change to the serial hardware interface. A lot of the UHS-I tricks aren't used for UHS-II. That possibly could apply to the Flash memory management too. It might just be a case of turning on caching and queuing and forget the other oddball modes, even for UHS-I.