Where to start with logging to SDcards? Updated 10/4: Success, correct pic

WBA Consulting

I now have a Parallax uSD module and will be plugging it into my demo board. Anyone have any recommendations on where to start with code?

I see a few different objects in the OBEX for SD, which is best to use? I was going to start with "FAT16 routines with secure digital card layer" by Tomas Rokicki since it has the most downloads. (wrong decision method?)

Any tricks warnings with structuring data for the SD objects or can I send anything to the SD objects?

I'll be logging time from an RTC, Temp/humidity readings from an SHT11, and RCTime values from a thermistor.

thanks in advance,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

Post Edited (WBA Consulting) : 10/5/2009 5:56:46 AM GMT

heater

There is a new fsrw here http://forums.parallax.com/showthread.php?p=824166

Not sure if it's made it into OBEX yet but it's been working fine for us.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Cluso99

PropCmd and PropDos have routines to write test data to an SD card under a FAT16 formatted card. You can modify this.

The latest drivers are on the above thread as heater suggests. There are test programs in this code for testing the SD card, so likewise, you cold modify this code.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

heater

No tricks for writing data to a file on SD.

I'd just write the values to the files as human readable numbers separated by commas. Use a new line for each entry with the time stamp as the first item.

This way it's easy to see if your data is recorded correctly, it's a text file, and you can easily import the data to Excel etc.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Oldbitcollector (Jeff)

I too would go for "comma delimited" data. As stated it'll give you the ability to easily import into Excel.

Here's a link to a little SD tutorial that Cluso and I did some work on a while back.
www.warrantyvoid.us/files/SD_trainer.zip

It's enough to get you started if you've never worked with the FSRW objects before.

OBC

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New to the Propeller?

Visit the: The Propeller Pages @ Warranty Void.

ElectricAye

Andrew,

This is probably obvious, but I suggest you think well in advance how you want your files named. For example, you can have the code from the OBEX give each data file a name and extension (.csv) so Excel will recognize it right away. You are limited in the size of the file name, so you might want to develop some kind of unique date or time or project code to help you locate or process files when you upload them from the SD card. Of course, you can always rename files after you upload them and stick on any extension you want, but if you've got tons of files, you might want to figure out a good strategy before you start taking data and find yourself awash in tons of files without any good "at a glance" sense of when they were generated.


my 1.1 cents worth,

have fun,

SamMishal

Hi WBA,
I find that the best way to learn something is in two ways
·· 1- See how it is done by others
·· 2- Do it yourself
I did this small test program to teach myself how to write and read data to and from a file
on an SD card using the propeller.
My test program uses P0-3 for the card....as suggested in the Docs for the uSD holder you
have.
The program creates and writes a file called Test.Tst on the SD card and then reads it
and displays the data on the Parallax Serial Terminal.
The program is very well documented and should help you in seeing how to:
·· - write/read Text variables
·· - write/read Longs/Word/Byte variables
to and from a file on the SD card.
Basically all operations you are likely to need.
The program uses the FSRW_23 given in the posting above. Also it uses FullDuplexSerial object to
do user I/O on the PST. Also FWR_23 itself uses mb_rawb_spi.spin object.
When you first run the program you will need to press Enter on the PST to start it running
the program·waits for you to make the PST ready and then press Enter.
By studying the code in the top level object SD_Card_File_Write_Read_Test.Spin you should get
a very good insight of how to use the excellent FWR_23 object to do all you need to do with an
SD card.
Make sure you connect the uSD holder you have as shown in·its documentation
· DO· = P0
· CLK = P1
· DI·· = P2
· CS· = P3··· also Vdd = 3.3 and Vss = Ground
I used the Propeller Demo board but you can use pretty much any board.
Do not forget to press enter on the PST once it is up and running after you
have compiled to the Propeller.
The program also has a method I made called MakeText() that helps you convert
a number into its decimal text format which makes it easy to write a number as text to
file on the SD card. The program also shows how you can write a number as a binary
and also how to read it back into the propeller and save it to a variable.
Text decimal numbers are good to write to the file if you are going to use the file in
another program like Excel or just want to read them using Notepad or some other editor.
Binary formats are better if you need to use the numbers back in the propeller
program or in another program that can read Little Endian formats of 32 bits
numbers or just binary bytes.
In any case the program shows how to do all that.

I hope this helps
Samuel

Post Edited (SamMishal) : 9/25/2009 4:25:51 PM GMT

WBA Consulting

Thanks to all for the wealth of advice. I am hoping to spend a lot of time this weekend between the PE Labs to strengthen my fundamentals and getting a rock solid foundation of SD card usage. Lot's of learning in store, not much playing. I'll post my progress and any notes.

@SamMishal: That's my plans: take a peek at what's available, then dig in. I will probably use the demo board as it and my 2" monitor take up minimal space and are battery powered. (less room needed to leave things setup). All the talk about longs/binary/endian/etc is the reason why I am going through the PE labs. My SPIN fundamentals are weak, but I can look at a demo program and tweak it for my needs (without fully understanding it). This is bad though when I need to create something that doesn't currently exist as a whole.

@OBC: Thanks, I'll run through those files as well. Nice to have different approaches to study.

@heater/electriceye/obc: My filenames are not critical as any data will be merged by excel and sorted/graphed using the time/date that will be part of the data. Comma delimeted was my thought and I would have been asking about that once I got things working, so thanks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

WBA Consulting

Lost a bit of my free time to a Windows 2000 Server that I had to pull out of service Friday night and Saturday afternoon, but was able to do a bit last night. Got the uSD wired up on my demo board setup and even modified my demo board case so it could be semi-permanent during my development. (the picture is with the lid off, but with the lid on, the uSD card has a nice little slot for access)
I loaded OBC's SD Trainer as a start and all I had to do was change the IO pins to 3-6 and everything worked first shot. I am now going through the FSRW code SamMishal's code to learn a bit more before I start merging my SHT11 code with the SD. Basic usage of an SD card is ridiculously simple.

Maybe it's a newbie thing but it seemed like it took me too long to find how to change the pin definitions for the FSRW object. I guess I was expecting that to be in a real obvious spot. Maybe I missed something in a readme somewhere and that's why it took me a while to find it digging through all the .spin files? It mentions looking in "sdspiqasm.spin" for connectivity, but I don't see that file anywhere. In "mb_rawb_spi.spin", it has this in the DAT block:

pinDO         long 0    ' pin is controlled by a counter
pinCLK        long 0    ' pin is controlled by a counter
pinDI         long 0    ' pin is controlled by a counter

I don't understand how to work with that yet. As I kept looking around, I also found this in the con block of "sysdep_demo_ser.spin" file:

sd_DO = 0
   sd_CLK = 1
   sd_DI = 2
   sd_CS = 3

But I don't see "sysdep_demo_ser.spin" being called out anywhere. Anyone have the simple answer to pin definitions for FSRW? It's ok if I am missing the obvious? I have to write up the supporting docs for the new server I just installed, but after that, I am back on uSD code!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

SamMishal

Hi WBA,
In the FSRW_23.Spin there are TWO methods: Mount· and Mount_Explicit
The Mount method takes 1 parameter which is the pin number of the 1st pin in the group
so you say Mount(0) to make it use P0,P1,P2,P3
Mount_Expicit(....) takes 4 parameters with which you can specify NONE CONTIGOUS pin numbers
You have to call ONE of these methods to initialize the communications between the propeller
and the SD card. You have to·do this·before you do any other actions.
Also call·the unMount method ·when you end using the SD card.
Well written objects provide METHODS for people to be able to configure and utilize the
functionality of the object.
You are not expected to go inside the object and start changing things inside it. You can do that
if you wish to modify the object to do something VERY different from what it was designed
to do.
Most well written objects provide METHODS for you to specify things like Pins to be used
Speed, and other specific things.
Also most objects have a Start() method that allows you to "start" the object's cog and
get it running and doing its work. Then later you can use other methods to set values or
get values.
In the FSRW object instead of a Start() method there are TWO methods more appropriately named.
This is a sign of a GOOD design. Providing different INITIALIZATION methods to provide for different
ways to use the object and also to allow for more versatility.
You will find that the FSRW object is WELL DESIGNED and it provides methods for almost any
operations you are likely to need and the object can support. So you will not need to go
into the guts of the object to change it.
The only quibble I had was it did not have a GREAT documentation.
So I changed that for myself. You will find the object FSRW_23.Spin in the archive I attached
earlier has a LOT of commented text at the top and you find a list of ALL methods you can use
listed in a good grouping order and a description of what they do and what parameters are
needed etc.
Regards
Samuel








Post Edited (SamMishal) : 9/29/2009 8:34:29 AM GMT

SamMishal

Hi WBA,
VERY NICE enclosure....I like it... it looks nifty too.
What is the module that is above the uSD card holder in the picture. It looks like it would be sticking out of the enclosure when closed?
One thing I am always worried about with the Propeller Demo Board is that whenever I am plugging or unplugging the PS/2 mouse and keyboard,·the force applied for the insertion/removal can exert TORQUE on the soldering points and break the connectors off the board.
Your enclosure solves the unplugging problem since the body of the enclosure provides leverage away from the bottom of the adaptors near the soldering pins. BUT... I THINK the plugging can still exert a torque on the solder points.
·
NICE WORK....did you make it yourself or just adapt a stock made one?
Samuel

WBA Consulting

SUCCESS!

Didn't have much time over the last few days, but pulled a late night because I couldn't get this out of my head. I modified my multiple SHT11 sensor code with some "excerpts" from OBC's SD_Trainer program. (see OBC's earlier post) This now runs the VGA display for two sensors, gives status data to the Propeller Serial Terminal, and logs the readings onto the SD card in a CSV format file called "SHT11.log".
I also made a second version with the debug terminal commented out because if the FTDI chip is unpowered, the prop hangs when starting the serial terminal (which meant it only worked when connected to my laptop) Both code files are attached.

SHT11_Multi_VGA_SD_002.spin is the main program
SHT11_Multi_VGA_SD_102.spin is the main program without debug
SD_Read_SHT11-log.spin is a modified version of SD_Trainer that reads back the SHT11.log file to the debug terminal
rest of files in zip are the required child objects

Debug terminal shows this while running (and just before clearing the screen to take another set of readings)
The "waiting" shows for 20 seconds, then the "starting in 5 seconds" gets added as a "don't pull out the card I am about to mount" warning for me. When only "waiting" showing, I know I can turn it off or pull out the SD card without corruption.

** Sending information to SD

** Routine Finished

** Sending information to SD

** Routine Finished

Waiting...
Starting in 5 seconds...

When running the SD_Read_SHT11.log program after a handful of readings, this is what's on the display:

Starting SD PlungeLogger Reader (SHT11.log)

** Reading information back from SD
Internal, 78.9, 42.4
External, 77.7, 42.2
Internal, 78.6, 42.6
External, 77.8, 42.1
Internal, 78.6, 43.4
External, 78.2, 43.8
Internal, 78.6, 43.0
External, 78.8, 41.8
Internal, 79.8, 41.8
External, 80.0, 40.2
Internal, 79.2, 43.7
External, 79.1, 42.4

** Routine Finished

My to-do list is taking shape now.....
1) Create a way to suspend data logging for safe powerdown/card removal
(check for status of an "enable" pin just prior to going to waiting mode?)
2) Create status LEDs for visual feedback
3) Move project from demo board to PropstickUSB and perfboard
4) Add Real Time Clock functions
5) Add functions (using DIP switch) for program choices: set time, log data, read out data through VGA or Serial port
6) Convert to manufactured PCB assembly
7) Waterproof it with a power source
8) Wear it while jumping into Lake Tahoe on 3/20/2010

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

WBA Consulting

The enclosure is stock case modified by me during my hunt for a low-cost Demo Board enclosure. All mods were done with a step drill, xacto, or nibbler, so it doesn't look too much like a hack job.
The top (or lid) is about 3" tall so there is plenty of room for the SHT11 module that is standing up on it. There will eventually be some type of display in the top so that I can have a full Propeller Development box in a small size. I can post some more details on it tomorrow.

Regarding the PS2 connectors: Almost all PS2 connector design has solder posts that provide sufficient support for plugging/unplugging. The ones used on the demo boards have this and are rather sturdy. Even so, I still support the back of the connector when plugging in. (safety first!)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

WBA Consulting

The numbers are temperature and relative humidity. I have two sensors currently on my demo board and just labeled them as if one would be in the case and one would be outside the case just so they would each have their own name.

For a little more insight to this project, see this thread:
http://forums.parallax.com/showthread.php?p=843197

The "plunge" is an annual event done at Lake Tahoe in Nevada to raise money for Special Olympics. I did it for the first time this year back in March and I am hooked. (I also did a local one in January that was done in a community pool filled with 1200 lbs of ice)
Follow the links in the thread above to see my crash-course BS2P40/USB Drive version I made. After getting into the propeller from UPEW, my goal is to have a full fledged prop based datalogger for next year's plunge. See this thread for a brainstorming I started on a PCB design (which is still in the works and will undergo another major change soon)
http://forums.parallax.com/showthread.php?p=842567

The link below my signature has info as well. Good cause, good fun, and a refreshing 42 degree swim. (I felt wonderful all day after the plunge)

this post was in response to a post by SamMishal, but for some reason that post disappeared. Anyways, just more detail for the overall project

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

Post Edited (WBA Consulting) : 9/29/2009 5:17:54 PM GMT

WBA Consulting

As I was going through my notes and posts to start on the next step, I realized the picture I attached previously doesn't actually have both sensors and the one there wasn't actually mounted in the circuit. Here's an updated picture.

Also, I ran it all night long to check things out and it worked great. The graph shows temperature and humidity of both sensors. The decrease in temp and increase in humidity in the initial portion of the graph is from moving it from the living room to the bedroom where the humidifier and ceiling fan were on. You can see that the two sensors did not measure exactly the same. The temperature averaged a 0.3 difference and the humidity averaged a 1.2% difference.

SHT11 modules have VDD, VSS, and CLK daisychained across the breadboard straight off the demo board header. Data lines are P1 and P2 respectively.
uSD card is using IO pins to 3-6 (spiDO=3, spiClk=4, spiDI=5, spiCS=6)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010