Programs for all boards

Rsadeika

The program attached is a softRTC, a clock program that runs in RAM, while the board is turned on. It is a UI based program, so you have commands to set the clock, and then get the time, and etc. This is a C version, although a C++ version should not be a very big effort to accomplish.

I am still thinking about a filetest program that could be run on boards that do not have an SD reader, although I am not sure how the upper 32KB could be set up to work with a file directory. Of course this kind of program would limit it to using the CMM mode, if the EEPROM is going to be used for storing files. Still not sure if something like this is possible.

Ray

Dave Hein

You could write a file driver for EEPROM. I did this a couple of years ago, but I abandoned it and used SD instead. I found that EEPROM just wasn't large enough to be useful. BTW, a lot of boards only have a 256 Kbit (32KB) EEPROM, so there isn't any additional space available. You probably need a 1024 Kbit (128KB) EEPROM to make it useful.

Rsadeika

Do you still have any of the code handy, that you are willing to share? Maybe I can also learn how to work the i2c too manipulate the EEPROM, plus something about file drivers.

Ray

Dave Hein

I might be able to dig up the code I had, but I think I would implement it differently now than I did the first time. I would probably implement it as a simple version of an SD FAT file system. The sectors could be 64 bytes in size, and a cluster size of 512 bytes. An 8-bit cluster index would handle a 128KB EEPROM. The FAT table would require 256 bytes.

jazzed

Dave Hein wrote: »

I might be able to dig up the code I had, but I think I would implement it differently now than I did the first time. I would probably implement it as a simple version of an SD FAT file system. The sectors could be 64 bytes in size, and a cluster size of 512 bytes. An 8-bit cluster index would handle a 128KB EEPROM. The FAT table would require 256 bytes.

Hi Dave,

What would it take to make a version up to 32MB for Flash?
I'm thinking of the case where 2x 16MB QuadSPI devices are used.

Thanks,
--Steve

Dave Hein

A flash file system would be nice. Flash blocks have to be erased before they can be written to, and I they have a similar rewrite limit like EEPROMs. I did a little bit of research on this, and flash file systems are typically log-structured. They are written as a long sequential stream of files. Any file that is rewritten is basically copied to the end of the stream, and the original version is invalidated. The file system has to be compacted when there is not much space left at the end, and the holes need to be removed.

The original EEProm file system I wrote is similar to this in that it was stream oriented. However, each block contained a ponter at the end that would chain to the next block. I'll have to look at it again.

jazzed

I guess wear-leveling comes up if the file-system gets thrashed a lot. Is it worth pursuing?
A big "don't abuse it" disclaimer and/or some limiting strategy would be needed for chip type EEPROM/Flash.

Dave Hein

I've attached a zip file that contains the EEProm file system I did a couple of years ago. There is also a Spin version of the filetest program to test it out. eefs.spin uses a similar API as FSRW with popen, pread, pwrite, etc. On top of that I have a layer with functions named hopen, hread and hwrite that use a pointer to a struct containing the state variables for each open file. cfileio.spin then provides a layer on top of that with the C stdio API. clibsd.spin provides other C functions, and pass-thru functions that just call the functions in cfileio.spin.

eefs.spin implements a file structure kind of like the FAT file system, except there is no central file allocation table. Instead, each block contains an index that indicates if the block is in use, and the address of the next block in the chain. Directory entries consist of 30-bytes each, which allows for 2 entries for each 64-byte block, plus the 4-byte index. For the most part, directories are treated just like files. The size is updated whenever a new directory entry is added to the end. This should probably be changed so that the filesize is zero for a directory, just like in the FAT file system. That would reduce the number of times the directory pages get rewritten. Also, the current method for allocating a new block is to just use the first one available. This could be modified to use a page write count, and use the block with the least number of writes instead.

Dave Hein

jazzed wrote: »

I guess wear-leveling comes up if the file-system gets thrashed a lot. Is it worth pursuing?
A big "don't abuse it" disclaimer and/or some limiting strategy would be needed for chip type EEPROM/Flash.

A flash file system is treated like a big circular buffer, where new and modified files are added to the end. A reclaim operation will copy files from the head to the tail of the buffer until the holes are all gone. I think this technique automatically performs wear-leveling. One draw-back to this technique is that the directory is distributed thoughout the file system. The filename, size and other attributes are included together with the file contents. This requires caching the directory in memory if you want to quickly search the directory.

jazzed

Nice EEPROM file-system demo Dave !

The circular buffer flash approach sounds good.

Rsadeika

I just tried the Spin version of the EEPROM filetest program, worked as expected. I also ran it through the spin2cpp program, for the ccode version it came up with an error complaining about the double use of malloc. For the cpp version it converted without any complaints, it also loaded and ran with SimpleIDE, but nothing came up on the screen. I am now in the process of building it in C using some of the spin2cpp converted files like eefs, pasm_i2c_driver, and cserial. Will see how far I get with this. Anyway, Dave thanks for finding the code.

Ray

Dave Hein

Most of eefs works OK, but there are a few strange things with subdirectories. They are treated like files, and filetest will do a "cat directory" without complaining. rm and rmdir don't distinguish between file and directories very well either. I think it has to do with how eefs orders the attribute bits versus how FAT does it. I'll clean that up at some point.

I encountered problems with spin2cpp and CLIB before. This is because CLIB contains methods that use the standard C function names. Eric got around this by capitalizing the first character of all the Spin methods when he converts it to C++. Maybe he neglected to do this when converting to C. Try changing "malloc" to "Malloc" in the Spin code to see if that fixes it. However, ultimately you would just want to use the standard C functions provided by the library. It would be nice if spin2cpp understood that calls to the CLIB methods should be preserved as calls to standard C functions.

Integrating eefs with the C file I/O will require duplicating the drivers used for SD file I/O. You'll need to modify FileDriver.c, which is located in the propgcc/lib/drivers source directory. You'll need to call functions in eefs instead of dosfs. You'll also need to modify the dfs_mount routine to make an eefs_mount routine. It's not a trivial task, but can be done.

I looked at flash file systems a bit more, and I think eefs could be modified to handle flash chips. It would need a few changes to handle wear leveling better, and it would need to be modified for the 512-sector size. It would also need changes to support block erase and reclaim.

Rsadeika

I ran filetest EEPROM through spin2cpp, the actual error is Multiple use of object cmalloc not allowed in C mode. I checked the Spin objects and clibsd.spin calls for a cmalloc, and cfileio.spin also calls for a cmalloc. So, I guess to get around this I would have to combine the two files, and then find replace the names.This does not sound like a lot of fun. Starting to think ...

Ray

Rsadeika

I started a rebuild of the filetest program for the EEPROM. So far all I have is some essential parts, so I can get a UI, and start testing functions like mount(). But, I have a small problem that I can not figure out, in the program. I expect that when I just hit the CR, I should get "Invalid Command" on the screen, that is not what is occurring, I just get ">". This must be something completely obvious that I am not seeing, can somebody clue me in?

Ray

/**
 * @file FTee.c
 * This is the main FTee program start point.
 */

#include <stdio.h>
//#include <stdlib.h>
//#include <string.h>
//#include <cog.h>
//#include <ctype.h>
#include <propeller.h>
//#include <unistd.h>
//#include <sys/stat.h>
//#include <dirent.h>
//#include <sys/sd.h>

extern _Driver _SimpleSerialDriver;
extern _Driver _FileDriver;



FILE *stdinfile;
FILE *stdoutfile;


/**
 * Main program function.
 */
int main()
{
    int num;
    char *tokens[20];
    uint8_t buffer[20];
    
    stdinfile = stdin;
    stdoutfile = stdout;
    
    waitcnt(CLKFREQ + CNT);

    while(1)
    {
        printf("> ");
        fflush(stdout);
        gets(buffer);
        num = tokenize(buffer, tokens);
        num = CheckRedirection(tokens, num);
        if (num == 0) continue;
        if (!strcmp(tokens[0], "help"))
            Help();
        else if (!strcmp(tokens[0], "quit")) break;
        else
        {
            printf("Invalid Command\n");
        }
        CloseRedirection();
    }
    printf("System Stop.\n");
    return 0;
}

Dave Hein

If you just hit CR the tokenize function will return a value of zero tokens, and the "if (num == 0) continue" statement will just jump back to the beginning of the loop. I've been working on converting eefs.spin to C. spin2cpp is a great program, but it produces C code that is somewhat obfuscated. I used another program I wrote to convert eefs to C, and I'm working on fixing up the parts where I used longmove to copy the calling parameters to local variables. I hope to have this working later today.

Rsadeika

I guess I will hold back until you get that completed, then I can work with a new C version of eefs. I noticed in the eefs file that it contains functions like format(), mkdir(), ..., etc, but I did not see a rmdir. I guess the original List() function will have to be rewritten to utilize the fuctions that are available in the eefs file?

Ray

Dave Hein

Here's a version of the filetest program called eetest that works with the EEPROM file system. I added a format command, so that the EEPROM can be reformatted. It uses the upper 32KB of a 64KB EEPROM. You can change the last two parms in the mount_explicit routine from 0x8000, 0x10000 to 0, 0x8000 if you want to use the first half of the EEPROM. This will work OK as long as you always run from RAM.

I'm still using the spin2cpp version of eefs.c. I'll generate a cleaner version of eefs.c at some point. Then the next step is to add wear-leveling support and make it work with flash memory.

Rsadeika

So, with this new eetest program, what would it take to make usable for all three systems, the SD, EEPROM, and flash? That way you could have one program, that could use whichever memory that you choose. Having not seen your program yet, how big would it grow if it handled all three systems?

Ray

Dave Hein

The functions fopen, fread, fwrite, fclose, fseek and remove are handled by function pointers in the driver structure, so those can be changed by adding the correct driver to the driver list. Normally, a file name will contain a prefix like SSER: for simple serial. However, the SD file driver does not us a prefix, and I used the same convention for the EEPROM file driver. This means they cannot be used together. We could add prefixes so that SD files could be accessed as sd:filename and EEPROM files might be ee:filename. This way they could be included together.

The directory functions are not part of the file driver. We would need to add them to the file driver function list to make them accessable in the same way as fopen, fread, etc. The drawback to this is that they will always be linked into the executable even if you never call them.

I tried a test where I added _FileDriver to the driverlist, and changed the File_prefix in eeFileDriver.c from a null string to "ee:". The program grew to about 26KB. I was able to copy a file from an SD card to EEPROM by typing "cat file1 >ee:file1. However, the opendir, mkdir, chdir, etc. functions only worked on the EEPROM due to the issue I mentioned above.

Dave Hein

Here's a new version of eetest. I cleaned up eefs.c so it is much more readable now. I also added a "-d" option to the ls command that prints the sector numbers used by a file. I still haven't added the wear-leveling code, but that's the next thing to do.

Rsadeika

I was just looking at your eefs Spin version, it is a nice little program. Do you happen to have a Spin version that uses the uSD? It would be nice to combine the two, ee and uSD, into one Spin version, I like your IO style.

I have a new project that I started, using the XBee and Sensirion SHT-11 module. I am thinking of having a remote slave board that has the SHT-11 and XBee on it, and the base board, with an XBee on it, which will transfer the info to a PC for processing with a spreadsheet or database. I tried working out some ideas using C, but I just can not get the create new cog in XMMC to work the way I want it too work. So, what I am thinking is to make version in Spin, if all the code fits, then do a spin2cpp on it, just see how it looks and works in C.

The slave board, as I mentioned will have an XBee and SHT-11 module, which will use the uSD for temporary storage before it gets sent to the base board via XBee. I could probably use the eefs Spin version to start, but I would really need to be using the uSD for longer term data gathering. Any ideas will be appreciated.

Ray

Dave Hein

I have been using an SD file driver for a couple of years that has a similar stdio interface. I've actually been working on cleaning it up for the past few weeks so I can post it in the OBEX. My latest version of the code is attached below. The SD file driver is located in cfileio.spin. The lower half of the file is a modified version of FSRW that supports subdirectories and multiple file instances. The top half of cfileio.spin contains the stdio functions fopen, fread, fwrite, etc. cfileio.spin is integrated with clibsd.spin to provide file access with a file stream pointer in the same way that the C functions work. You can also use cfileio.c plus cmalloc.c without the rest of the CLIB objects.

An early version of spinix supported both SD and EEPROM file systems at the same time. The two file drivers were mapped to different directories under the root directory so you could do something like "cat /sd/file1 >/ee/file1". I removed EEPROM file support from later versions of spinix.