Variable SD Card names in FemtoBasic?

Duane C. Johnson

Hi Mike;

I want to be able to read many thousands of data files on an SD Card using FemtoBasic.
The file names are basically numbers.
However, the command description requires names to be a "string constant". Or am I wrong?

This doesn't work for me. Yes, I can work with a single huge file that contains all the data.
But it takes to much time to access an element in the file. The whole data set may contain as much as 1G bytes or so. This data set is to contain solar position data for 30 years or so. Well actually the stepper motor values for the 30 years.

It would work so much better to have many files each representing an element that is quickly found, opened, and read.

Is there a way to use something other than a string constant?
Or a way to change or fudge a string "constant"?

Thanks
Duane

Mike Green

The things that look like string constants are actually scanned directly from the source code line, so there's no way to fudge a string "constant". You'd have to modify the FemtoBasic interpreter to get what you want, but it would be pretty easy to do. Do keep in mind that the FemtoBasic uses Rokicki's SD card driver which doesn't support subdirectories. As a result, you're limited to about 64 files in the root directory, so your scheme wouldn't really work.

It would be possible to modify Rokicki's driver to allow random read file positioning and add a statement to support it, but it would take a bit of work. It would also be possible to allow an optional numeric suffix for a file name in the OPEN statement. I've mentioned before that I don't intend to add any more features to FemtoBasic. It's already a large program. You're welcome though to add features to make your own version. I'd be happy to make suggestions and offer advice.

Mike G

What if you had two files. The first file contains two columns, a unique index (or hash) and a pointer to the record in the second file. It makes it easier if the record lengths are equal. Search the smaller file first to get the record position in the second file. Then open the second file and go straight to the record. The search could be fairly fast if you are only looking at every other 32 bit field in the first file. If your records where 128 bytes long then you could have 0xFFFF_FFFF/128 = 33,554,431 records or the max file size.

ID         Pointer
------------------------------
0x00000001 0x00000000
0x00000002 0x00000080

You can use a hash for text type searches but the hash index has to be unique. Julian date could be the index too.

rokicki

Mike Green wrote: »

It would be possible to modify Rokicki's driver to allow random read file positioning and add a statement to support it, but it would take a bit of work.

The current release of my driver has supported seeking (random read file positioning) for some time.

-tom

Mike Green

In the version of Rokicki's driver used in FemtoBasic, there is a random read seek routine, but it's able to seek only within the current file cluster which is limited to 32K. That wouldn't work for this application. If there's a newer version that supports wider seeks, I'd have to adapt it to work with FemtoBasic. Alternatively, I could try to use Kye's driver in FemtoBasic. Either would take a bit of work and it's not at all clear that Kye's driver would fit without severely impacting the amount of memory available for the Basic program.