filetest.spin echo and redirection

Rsadeika

Based on the filetest.spin program, I added a Sensirion, and XBee module(s). What I am trying to do is datalog the temp data as it gets streamed to the master program. I created a getxb command, which captures a slice of the streaming data. When I try getxb > echo > test1.txt, I have two files created, but both are empty of data. I tried different combinations, and the best that I get are some empty files that are created. So, my question is, how can I get the captured slice of data inserted into a file, using getxb as the command? I know getxb works, because when I use getxb, I get the slice of data shown on the console screen.

Thanks

Ray

Dave Hein

Assuming you implemented getxb like the other commands you don't need to specify "> echo". You just need to type "getxb >test1.txt". Also, the getxb command needs to use fprintf, and not printf. Look at the code for the echo command, and use the same technique for printing from the getxb command.

Rsadeika

OK, thanks, I got that fixed. The other question I have is, when I try to run the program, via an external program like Tera Term, instead of the console terminal, instead of doing a NL, it does a CR, it stays on the same line when it prints the char stream. In other words if at the ">" I type in help, it prints out the contents of "help", and does not drop down a line, with a new ">". I have two different terminal programs, and the behavior is the same for both of them. Is their something that you added that is specifically used for the SimpleIDE console terminal? I checked the code, and I did not find anything that would make it act differently when using another terminal program, maybe I am missing something?

Ray

Dave Hein

I think in that version of the code I use a "config" variable that controls whether a newline transmits a CR, LF or both. By default it will only send a CR. You can call c.setconfig to change the value. The CR and LF are mapped to the two least significant bits of the config value, so c.setconfig(2) will cause newlines to send only a CR, c.setconfig(1) will send only a LF, and c.setconfig(3) will send both. It sounds like you need to call c.setconfig(3) when using Tera Term. I'll document this once I get around to updating CLIB in the OBEX.

Rsadeika

Another item I can not figure out, how to get a decimal input from the console IO? I have implemented the Seico RTC driver, I am using the DNA-RTC board, and I am trying to get number(s) input, like 9, 12, 18, or 23. I tried the scanf1, fgets, gets, and the others, just can not find the right combination to capture a decimal input.

On another note, I tried to run my modified filetest.spin with spin2cpp, and it came up with an error, so not everything can be resolved using spin2cpp.

Ray

jazzed

Rsadeika wrote: »

Another item I can not figure out, how to get a decimal input from the console IO?

One of several ways below. One can also use fgets and atof/strtod.

/**
 * @file scanf.c
 */
#include <stdio.h>
#include <propeller.h>
/**
 * Main program function.
 */
int main(void)
{
    float f;
    waitcnt(CLKFREQ+CNT);
    printf("Enter a float: ", f);
    fflush(stdout);
    scanf("%f", &f); // must pass address of the variable to fill
    printf("You entered %f\r\n", f);
    return 0;
}

Rsadeika

These questions are for Dave Hein's testfile.spin, I am in Spin mode at the moment. Actually I was looking for an atoi equivalent in the clib object, but I guess there is none. Still have not figured out how to resolve the above problem.

Ray

jazzed

Rsadeika wrote: »

These questions are for Dave Hein's testfile.spin, I am in Spin mode at the moment. Actually I was looking for an atoi equivalent in the clib object, but I guess there is none. Still have not figured out how to resolve the above problem.

Ray

You said scanf didn't work; I showed you how it works. The atoi function is in our library; I use it in the simple calculator example. I'm sure Dave will be happy to address other issues.

Edit: I use atof in the simple calculator example. It and atoi are in the library and work fine of course.

Dave Hein

Rsadeika wrote: »

Another item I can not figure out, how to get a decimal input from the console IO? I have implemented the Seico RTC driver, I am using the DNA-RTC board, and I am trying to get number(s) input, like 9, 12, 18, or 23. I tried the scanf1, fgets, gets, and the others, just can not find the right combination to capture a decimal input.

On another note, I tried to run my modified filetest.spin with spin2cpp, and it came up with an error, so not everything can be resolved using spin2cpp.

Ray

You should be able to use scanf1. You need to make sure you pass a pointer to the value your reading into, such as c.scanf1(string("%d"), @tempval). There are some issues with converting clib.spin to C using spin2cpp. I forget what the specific problem is, but I recall running into it before. Of course, what you really would want to do is convert the rest of your code to C, and replace the calls to the CLIB functions with the standard C calls.

Rsadeika

I got the scanf1 to work the way I want it to. Now the next little problem:

Var
  long logstack[40]
PUB start
  logmode := 0
  cognew(logger,@logstack)

PUB logger
   
  repeat while(logmode)
    Date >> logfile.txt
    Time >> logfile.txt
    getxb >> logfile.txt

Using the above general outline, I would like to have the logger start data logging when logmode gets changed to 1, and stop when it gets changed to a 0. So, far I am unable to get logfile.txt to appear as a valid file name, compiler keeps pointing this out as an error. I thought that the redirection would work within a PUB, but I guess it only works in the console IO mode. Any suggestions as to how I can get this to work as I would like?

Ray

Dave Hein

The "command >> outfile" syntax is interpreted by the code in the main routine. In theory, you could break out the code that does the parsing and calls the appropriate function, and put it in a separate function. You could then call the function with something like "execute(string("Date >> logfile.txt")), but this would be a lot of extra work, and an inefficient way to do what you want to do. What you should do is open logfile.txt before you start the logger cog, and then have the Date, Time and getxb routines write to the file.

You should be able to do the following:

Var
  long logstack[100]
PUB start
  logmode := 0
  stdoutfile := c.fopen(string("logfile.txt"), string("a"))
  cognew(logger,@logstack)

PUB logger
   
  repeat while(logmode)
    Date
    Time
    getxb
    c.fflush(stdoutfile)

Note that the file routines in CLIB aren't cog-safe, which means you can only do file I/O in one cog. You could provide your own locking mechanism to make it cog-safe. I also added the call to fflush to ensure that the data get written out to the log file. This wouldn't be necessary if you can ensure that you will cleanly close the file when you shut down.

Rsadeika

I have included both the master and slave programs, this is SPIN that we are dealing with. I put in my interpretation of the logger function, which does not work correctly. The newcog starts up, but it is not working as expected, as it runs Date, Time, and getxb it shows the commands on the screen as it cycles with the repeat loop, but there is nothing that gets put into the logfile.txt. I was surprised to see the stuff on the console screen, I thought that the cog was running in the background. I not sure as to what is going on.

I added the getxb, setclock, settime, date, and time functions which work as expected on my DNA-RTC board. The slave0 board which has the sensirion module also works as expected. So, the code below works mostly, except for the logger part, and I need more hints as to where I went wrong, to get it too work correctly.

Ray

Dave Hein

You should call c.start before you use any of the other CLIB functions. c.start initializes the malloc routines, which are used by c.fopen. You also need to call mount before you call c.fopen.

You can't use stdoutfile for the logfile handle if you are also going to run the code in main. So you should declare a "logfile" variable and use that to write to the log file. This means you'll need to change Date, Time and getxb to write to logfile instead of stdoutfile.

As I said before, the file I/O routines in CLIB are not cog-safe. This means that you can't do file I/O from multiple cogs unless you implement a locking technique. I'll probably add cog-safeness before I update CLIB in the OBEX, but for now I would suggest that you avoid doing file I/O from multiple cogs.

Rsadeika

Thanks, Dave, for taking the time to look at my program, and making some comments. Now, I have to rethink what it is that I really want to accomplish, and if I want to make this any more complicated than it is. It seems that the direction that I am leaning too, is trying to convert filetest.spin into a very, very small OS, not sure if that would be a good idea. Do you have anything, for public use, that would address those needs?

I downloaded and looked at spinix, because it is a large program, I was unable to figure out where to start, too get it too run, on SimpleIDE. I tried shell.spin as a starting point, but did not get anywhere, so it must be another starting point. Since getting my FTm&s program functioning in a very minimal capacity, this might be a good time to explore some ideas about a very minimal OS that uses Spin, and then maybe it would be possible to have a matching version in C. IMHO, with the availability of modules and sensors like the XBee, BT, Sensirion, and others, it seems like a small OS is a necessary thing to have, in order to interact with the modules and sensors. Its either that or creating a program on the PC for controlling and maintaining data flow, which sounds to me like a little overkill for a project, or projects of this size, but I could be wrong. Any ideas about this?

Ray

Dave Hein

spinix apps use a modified version of CLIB called clib_os.spin. It's very easy to port programs that use clib or clibsd to spinix. The spinix OS itself is large because it includes various standard linux utilities, such as ps, od, cp, diff, ed, vi, unix2dos, dos2unix, more, etc. I'll put together a minimal set of files that just contain the binary files plus the source files that you would need to build an app.

Dave Hein

Ray, I looked into using spinix for what you want to do. However, there's not enough memory to run your app along with the OS kernel and shell at the same time. It's also probably an overkill for your application. I decided to go ahead and add a lock to the file code, which allows you to call it from multiple cogs at the same time. I also modified the logger routine so that it uses a file handle called logfile instead of stdoutfile. I modified the Date, Time and getxb routines to used a file handle parameter. I also added a couple of debug lines to getxb so that it returns after one second. This is useful for testing purposes. These two debug lines are currently commented out. I have attached updated versions of filetest.spin and cfileio.spin.

I ran some tests on the code, and the logfile.txt file grows once per second as expected. You can run the other commands while the logger cog is running. Let me know if you have any questions.

Rsadeika

Thanks Dave.
I just ran the program with your two new files, unfortunately it gets to "Load and mount SD:" , and just stays that way. I guess it is not reading or sensing the SD card. I ran the C version of filetest to see if the SD card was corrupted, but that program reads the card OK, so it's not the card. I also tried moving the mount command from Start down to main, and that made it worse it never got to the "Load and mount SD:". I am not sure where the problem is, maybe the new cfileio.spin?

Ray

Dave Hein

I forgot to change the call to mount_explicit back to the one you were using. I was testing on a C3 board. Comment out the one that I was using, and uncomment the one for your platform that uses pins 0, 1, 2 and 3.

Rsadeika

I just gave the program a test run, below is what showed up on the console screen. After I hit return on the startlog command, it did a newline as expected, but it started to do a newline after that. As you can see it excepted the stoplog command, which stopped the logging process. In the logfile itself, the first line is "This is the logger 0", not sure how that got in there. It looks like the first Date command is correct, but the Time command gets corrupted, and then both get corrupted after that. The getxb command data seems to not to get corrupted. The getxb command is '%s' string data, while the Date and Time command are '%d' data. Not sure why the logger cog is having difficulty with the '%d' data. But, other than that, the program is starting to work as expected.

Ray

> startlog
>


st
opl
og

>
cat logfile.txt
This is the logger 0
10/11/12
125:165:165
F 70.7 H 52.2165/165/165
125:165:165
F 70.7 H 52.1165/165/165
125:165:165
F 70.7 H 52.1165/165/165
125:165:165
F 70.7 H 52.1165/165/165
125:165:165
F 70.7 H 52.1165/165/165
125:165:165
F 70.7 H 52.1165/165/165
125:165:165
F 70.7 H 52.1>

Dave Hein

Looks at the prints in the getxb routine. The newline is coming from a printf0 while the other print is going to the logfile. Also, the first print in the logger routine is going to the logfile.

I suspect the problem with the date and time is happening when update is called from the logger routine. The first date print is OK because it is obtained before update is called. Since you are calling the I2C routines from two different cogs they might be both controlling the data and clock lines. Maybe cog 0 still have control over the lines when the other cog is trying to use them.

Rsadeika

The code seems to run the way I want it, but I am not sure if I just created some other hidden problem. I guess you were right, it was, or at least I think it is, an access of the I2C. I still have not tried out the setclock, and setdate commands, I am just happy I got this far.

Ray

PUB start
  logmode := 0
  c.start
  ' Wait for the serial terminal to start
  waitcnt(CNT + CLKFREQ*3)
  mount
  logfile := c.fopen(string("logfile.txt"), string("a"))
  cognew(logger,@logstack)
  c.setconfig(3)           ' Runs correctly with external terminal
  xb.Start(27, 26, 0, 9600)
  xb.AT_ConfigVal(string("ATMY"),$0)
'  seico.start
'  I2C.Initialize(28)

  main

PUB logger
  seico.start
  I2C.Initialize(28)
  waitcnt(CNT + CLKFREQ*3)
'  c.fprintf1(logfile,string("This is the logger %d\n"),logmode)
'  c.fflush(logfile)
'c.printf0(string("This is the logger\n"))
    repeat
      Date()   
      repeat while(logmode)
        Date(logfile)
        Time(logfile)
        getxb(logfile)
        c.fflush(logfile)

PUB getxb(outfile) | xbinput, newline
    'Uncomment the next two lines for testing
    'waitcnt(clkfreq+cnt)
    'return
    
    xb.RxFlush
    repeat while (1)
      xbinput := xb.rx
      if (xbinput <> "!")
'        c.printf1(string("%c"),xbinput)
        c.fprintf1(outfile, string("%c"), xbinput)
        next
      elseif (xbinput := "!")
        xb.RxFlush
'        c.printf0(string("\n"))
        c.fprintf1(outfile, string("%d\n"),newline)      
        quit

Rsadeika

Now that I have the new sleep function working I would like to run a 24 hour, one minute capture, data log session. The question is, how do I view the log file? Would the cat function get over whelmed with the size of the file? Probably I need to think about some method(s) of making sense of the data. I am not sure if the Propeller would be up to the task of hosting something like a grep command, I think that is what is used for data analysis? I guess I need some ideas about this.

Ray

Dave Hein

The cat function can handle large files, but it may take a while to print really large files on the screen. You could implement a tail function where you do something line "tail -20" to print the last 20 lines. A simple grep function would also be easy to implement. "more" may also be useful, so that you could view the file one screen full of lines at a time. Look in the spinix/src directories, and you'll see implementation of those programs. They use glob.spin so they can the user can specify multiple files using wildcard characters, but you do need that for your application.

I'm now rethinking whether you would be better off using spinix. It would provide all the utilities you need, and they don't use up hub memory until you run them. There's something like 32 apps in the spinix/bin directory that average about 5.5K each. There's no way all of these functions could be included into a single program that would fit in the 32K hub RAM. That's the advantage of an OS than can load an app from the SD card. I'll play around with your logger program some more to see if it can be run under spinix.

Rsadeika

Thanks Dave, I want to present a brief as to where I might be heading with all this. I will be adding/activating some more slave units. The master will have to deal with obtaining the data from these units, the logger program may have to change, but I am not sure. Right now the logger is sufficient when I have one slave streaming data, but I am not sure how that would be handled if I have two or more slave units collecting and streaming data. I was hoping that I would have been able to do a spin2cpp on the master, and maybe try expanding the C code to fit my needs.

Now, the thing I would have to think about is whether I want to have the slave units collect and store the data locally, then the master could, as a scheduled event, ask for the data to be sent to the master. With this scenario, it introduces a new problem, sending a file, which contains data, to the master. I have not seen anybody working on some File Transfer Protocols for the Propeller. In fact that opens up some more ideas, from the master I could instruct/schedule a data collection event to occur on the slave, then the slave, once it is done, could automatically send the data to the master. These are just a few ideas that I have in mind, how feasible, I do not know.

Ray

Rsadeika

I enhanced the startlog, stoplog, and quit commands. Before I added the cogstop, in the quit command, I noticed that when I quit, the logger cog was still running, now it stops the logger cog when I quit. I also noticed that the cog number for the logger is 3, that means that there is also a 1 and 2 cog session running? When I added a cogstop 1 and 2, and when I do a quit, it corrupts the uSD card, in a way, that when I do a restart, the program no longer recognizes the uSD card. I am not sure why that is occurring, so I do not do a cogstop 1 and 2 anymore.

Ray

Load and mount SD: done.
Commands are help, cat, rm, ls, ll, echo, cd, pwd, mkdir, rmdir, chmod
getxb, date, time, setclock, settime
startlog, stoplog, quit.
> quit
Cog number 3 stopped.
System Stop!

    elseif (not c.strcmp(tokens[0], string("startlog")))
      logfile := c.fopen(string("logfile.txt"), string("a"))
      logmode := 1
    elseif (not c.strcmp(tokens[0], string("stoplog")))
      logmode := 0
      c.fclose(logfile)
    elseif (not c.strcmp(tokens[0], string("quit")))
      cogstop(cNumber)
'      cogstop(2)
'      cogstop(1)
      quit

Dave Hein

You're actually running 5 cogs. They are as follows:

cog 0: main program
cog 1: CLIB serial I/O from c.start
cog 2: SD SPI driver from mount
cog 3: logger cog
cog 4: FullDuplexSerial from xb.Start(27, 26, 0, 9600)

You should tell the logger cog to stop logging, and then wait a fraction of a second before stopping the SPI driver. I suspect calling unmount would do an orderly shutdown of the file system.

As far as file transfer protocols, take a look at sb.spin and rb.spin in spinix/src. sb sends a binary file using the YMODEM protocol, and rb will receive a file. I use this with the Windows hyperterminal program to transfer files between my PC and the Prop's SD card.

Rsadeika

I just did a three hour logger session, and everything seems to be working as expected. Since I am using Tera Term, I was able to do a cat on the logfile, and then highlight and copy to a txt file on the PC. So there is a relatively easy way, short of taking uSD card out and reading on the PC. Now, I have to figure out how to prep the data that is being logged, so I can load it into a spreadsheet. Since I use OpenLibre, I think I should be able to manipulate the data, charts and that sort of stuff.

I think that I will have to redo the slave0 program, using filetest.spin as the basis for IO, in fact maybe I will start calling it ftOS. I just found out that there is a SpinLMM available, will have to check that out to see if there would be any advantages with using it with the redo of the slave0 program. The only problem is, I noticed that SpinLMM can only be used with bst or homespun. I do not want to use bst anymore, and I only use 64 bit Win 7 OS, will the SpinLMM work or compile with SimpleIDE, without any problems?

I would also like to redo my master program in C, but cannot figure out how I could get an equivalent of the logger cog to work in C. I tried once before and I could not seem to get an FDS in main(), and to have an FDS, for the XBee, to work in a seperate cog at the same time. It seemed like it had to be one or the other, but not both at the same time. If I can figure that out, then maybe I can start a rework of my master program.

Ray

Rsadeika

Just for the heck of it, I decided to try SpinLMM DEMO with SimpleIDE, no, it did not work.

Ray

jazzed

Rsadeika wrote: »

Just for the heck of it, I decided to try SpinLMM DEMO with SimpleIDE, no, it did not work.

Ray

Please offer some details about what did or did not happen. Got a link to the demo?

Rsadeika

It is the demo that is in the attached zip file, it is the SpinLMM demo. It did compile, which is a good thing I suppose, but that was about it.

Ray

jazzed

Rsadeika wrote: »

It is the demo that is in the attached zip file, it is the SpinLMM demo. It did compile, which is a good thing I suppose, but that was about it.

Ray

Hard to tell what the baud rate is supposed to be from the demo.spin code.
SimpleIDE default baud rate is 115200, and seems like the demo is using 57600.

I get this:

Commands
--------
float inum
round fnum
trunc fnum
add fnum fnum
sub fnum fnum
mul fnum fnum
div fnum fnum
Enter Command: add 1.2 1.4
add(1.2, 1.4) = 2.600000e+00
7712 cycles
Enter Command: 

So, I guess it worked for me.