FlexProp: a complete programming system for P2 (and P1)

pik33

In Windows, if I don't close the terminal before running the program again, I got this

Could not find a P2 on port \\.\COM5                                            
Press enter to continue...    

If, after this error, I run the program again, it loads and works. Then doesn't work. Then works... Every second run. This I have on both my Win11 computers. I have to either remember to close the terminal, or click "compile and run" (or use ctrl-r) twice.

evanh

@pik33 said:
Adding dim i in one or both of subs restores the proper behavior, so it is to remember, always declare your variables or strange things can happen .

Hehe, been there. I'm working with a multitasking Basic on an embedded platform at the moment. It has similar ill-defined variable declarations. It doesn't have functions() at all within a "program" task so variables within a program are sort of global but are not shared across the multiple programs. There is special reserved/fixed global space for that and they are all 64-bit floats only.

Functions do exist but must be placed in a "library" program as a collection. Each library has its own library wide globals and also can DIM locals with in each function. Functions can accept passed variables via the function parameters and have one return parameter. Otherwise only the fixed 64-bit global space can be shared, as per regular the program tasks.

__deets__

I think I found a problem in the file system implementation writing to the SD-card. This is the relevant code:

void write_sensor_data() {
  char line_buffer[100];
  char filename[100];
  find_filename(filename, sizeof(line_buffer));
  printf("found filename: %s\r\n", filename);
  FILE* testf = fopen(filename, "w");
  assert(testf);
  int readings = 0;
  ssize_t write_pos = 0;
  ssize_t buffers_written = 0;
  for(;;)
  {
    while(ringbuffer_empty(&g_adc_ringbuffer)) {}
    adc_reading_t reading;
    ringbuffer_pop(&g_adc_ringbuffer, &reading);
    int rb_count = ringbuffer_count(&g_adc_ringbuffer);
    ++readings;
    // Includes the final 0, so substract 1 off it!
    ssize_t bytes_written = snprintf(&line_buffer[0], sizeof(line_buffer), "D:%08X:%02X:%04X:%08X\n", reading.timestamp, reading.mux, reading.value, rb_count);
    bytes_written--;
    if(BUFSIZE - write_pos < bytes_written)
    {
      fwrite(g_output_buffer, 1, write_pos, testf);
      //printf("Wrote %i\r\n", write_pos);
      write_pos = 0;
      buffers_written++;
      if(buffers_written % 10 == 0)
      {
        printf("Update metadata %s\r\n", filename);
        fflush(testf);
        #ifdef FLUSH_WITH_CLOSE_AND_OPEN
        fclose(testf);
        testf = fopen(filename, "a");
        assert(testf);
        #endif
      }
    }

    memcpy(&g_output_buffer[write_pos], line_buffer, bytes_written);
    write_pos += bytes_written;

    if(readings % 1000 == 0)
    {
      printf("samplerate: %i, wrote %i, rb_count: %i\r\n", SAMPLERATE, readings, rb_count);
    }
  }

  fclose(testf);
};

The problem is in the if(buffers_written...) clause. Just flushing will not produce a file with actual data. The name shows up, but the file is empty. Only if I close and open the data will be accessible later via SD-Card.

I haven't yet investigated deeper than verifying that fflush calls __default_flush. My current assumption is that closing updates some metadata in the directory data structure establishing a link between the entry and the actual clusters of data. But that's just guesswork.

Any insights?

evanh

I never tried fflush() but fopen()/fwrite()/fread()/fclose() worked well back when I was doing the speed tests for adding performant smartpin support. So I guess that supports your thinking.

__deets__

I dug a bit deeper, and found that flush isn't implemented for FAT, but f_sync seems to do the trick (called as part of f_close).

I'm interested in robustness. Let's just say a rocket engine test stand can be subjected to rather violent forces, and retaining the gathered test data is very important..

Wuerfel_21

I guess there's a missing wrapper function.

I actually use fflush for SRAM backup in megayume and I don't think I ever had missing data, but that's re-using the file it created once and only reading/writing one big blob of binary data.

I see that f_sync should do it, will submit a PR to implement the VFS flush function with it.

Wuerfel_21

PR here: https://github.com/totalspectrum/spin2cpp/pull/389

Christof Eb.

@"Christof Eb." said:
@ersmith
Is there a chance, that you would like to enhance the features for those "external" global cog variables?
Reason to ask is, that I still feel tempted some times to try again on this MC6809 Coco3 OS9 emulator written in C. For it's cpu registers it uses:

typedef union
{
  unsigned short Reg;
  struct
  {
      unsigned char lsb,msb;
  } B;
} cpuregister;

This way the registers can be accessed as 8bit and as well as 16bit.
Just an idea....
Christof

@ersmith said:

@"Christof Eb." said:
@ersmith
Is there a chance, that you would like to enhance the features for those "external" global cog variables?

I'd like to do that, but it may take some time to write the code for the cases where only part of the 32 bit registers are used (especially on P1 it's a pain to extract bytes from COG memory locations).

Hi Eric,
I have to admit, that I do not know, if this would boost my capabilities and endurance sufficiently to enable this project, but I just wanted to ask, if You had the chance....
Christof

pik33

Another glitch.

I changed the function declaration but forgot to change the line

if part(endpos+1)<>")" then return 0,0.0,nil,-1 else return eresult,endpos+1,rtype

The proper return is after 'else' (the function returns 3 results). The not proper (old) return is before else (4 results). This caused a compilation error, as it should. What is a glitch, however, is the error message:

D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic002.bas:279: error: incompatible types in return: expected list of values but got list of values

So you got what you expected, why do you complain?

Should be: expected list of 3 values, got list of 4 values.

__deets__

@Wuerfel_21 danke! Pretty much the same I did, and I can confirm that works. You are better at the whole error handling obviously. Looking forward to the next release.

ersmith

@pedward : thanks for the RPM support offer, but I'm not really keen on trying to implement all of the various Linux distribution formats.

@pik33 said:
I made a mistake and wrote, in Basic

len(line$-c)

The special case code for BASIC strings handling + and - was somewhat broken. I think it should be much better now.

@pik33 said:
Another glitch. I don't know, bug, or feature If feature, needs explaining in doc.

>

This test code:

sub a()
for i=1 to 20
b()
print i
next i
end sub

sub b()
let i=i+1
end sub

a()

The problem is that let i = i+1 does not declare a new variable, because i+1 is on the right side. What it does is indicate that there is a global variable i that you want incremented. Since there is a global variable named i, the for loop uses that instead of creating a new one. This is all working as intended.

@Wuerfel_21 said:
PR here: https://github.com/totalspectrum/spin2cpp/pull/389

Thanks, I've merged that.

ersmith

@"Christof Eb." said:

@"Christof Eb." said:
@ersmith
Is there a chance, that you would like to enhance the features for those "external" global cog variables?
Reason to ask is, that I still feel tempted some times to try again on this MC6809 Coco3 OS9 emulator written in C. For it's cpu registers it uses:

typedef union
{
    unsigned short Reg;
    struct
    {
        unsigned char lsb,msb;
    } B;
} cpuregister;

This way the registers can be accessed as 8bit and as well as 16bit.
Just an idea....
Christof

@ersmith said:

@"Christof Eb." said:
@ersmith
Is there a chance, that you would like to enhance the features for those "external" global cog variables?

I'd like to do that, but it may take some time to write the code for the cases where only part of the 32 bit registers are used (especially on P1 it's a pain to extract bytes from COG memory locations).

Hi Eric,
I have to admit, that I do not know, if this would boost my capabilities and endurance sufficiently to enable this project, but I just wanted to ask, if You had the chance....
Christof

That kind of use of extern register isn't completely working yet. You can declare variables that way, but only the first element of each struct works properly. However, with a small modification you could get something that works:

typedef union {
    unsigned short Reg;
    struct {
//        unsigned char lsb, msb; // does not work for "extern register"
        unsigned lsb:8;
        unsigned msb:8;
    } B;
} cpuregister;

extern register cpuregister X;

int main() {
    X.Reg = 0x1234;
    __builtin_printf("Reg = %x lsb = %x msb = %x\n", X.Reg, X.B.lsb, X.B.msb);
    return 0;
}

avsa242

Hi Eric,

Do you think it would be possible sometime in the (even far) future to have object pointer syntax something along the lines of:

' Parent object main.spin
obj
    child: "gui.things"
    display: "display.driver"
    'display: "other.display.driver"    ' these two included for a visual hint that they could be swapped in place of the first one
    'display: "yet.another.display.driver"

pub main()
    child.init(@display)    ' syntax so far is the same as current FlexSpin

' Child object gui.things.spin
obj display = inherit display    ' or something similar...this would behave like the object definition in current versions of FlexSpin, but instead creates it automatically based on the obj symbol in the parent with the same name 'display'

var
    long _instance

pub init(optr)
    display[_instance].some_display_method()

I hope you can see what I'm trying to describe above - basically what I'm running into is a problem like what @jrullan described in his GUI thread before he implemented object pointers. I can define a pointer to a specific display driver object in the child with the current FlexSpin implementation, but what if I wanted to make it display driver-agnostic? I typically like to think of child objects/drivers for things as "read-only", so while I could edit the child object to point to a different display driver for each case, it seems like kind of a dirty way to do it. Having it be able to grab it from the parent somehow would be more elegant, I think. The only other way I can think of to accomplish something like this now is a bunch of function pointers.

Cheers,
Jesse

Wuerfel_21

@avsa242 said:
The only other way I can think of to accomplish something like this now is a bunch of function pointers.

Another current option would be to

' Child object gui.things.spin
obj display = CHILD_DISPLAY

var
    long _instance

pub init(optr)
    display[_instance].some_display_method()

and then put -DCHILD_DISPLAY=\"yet.another.display.driver\" into your build command.

Actually, I'm not sure if it likes having quotes in the define string...

avsa242

Ahhh that's true...didn't think of that, thanks

EDIT: Yes confirmed, defining a build-time symbol like that works for current FlexSpin versions, thanks again

ersmith

Maybe we could introduce string literal constants and then use the new constant override feature to do something like:

' main.spin2
con
  DRIVER_NAME = "my.display.driver"
obj 
  display: DRIVER_NAME
  child: "child" | DISPLAY_NAME = DRIVER_NAME
...

and in the child:

con
  DISPLAY_NAME="default.display"
obj
  display = DISPLAY_NAME

rogloh

That approach seems cleaner Eric. But how much of a change are string constants in general, and what are the implications on these constants in expressions? Will anything break etc?

I'm all for language/build improvements that help create more maintainable software projects as long as they won't mess up old code. Anything that prevents multiple copies of files with different hard copied values will be of use to me.

Maybe these string constants passed down could also eventually be used for "include file" names in objects in order to patch in more custom values and code than the OBJ reference in the parent could realistically be expected to pass down? Although we don't have any include files in Chip's language version yet as I recall. Does flexspin already support include files by the way? EDIT: just checked the docs, yes it does.

ersmith

@rogloh said:
That approach seems cleaner Eric. But how much of a change are string constants in general, and what are the implications on these constants in expressions? Will anything break etc?

The syntax I proposed won't work, because of the existing use of single character strings in constants (and the pervasive use of strings to mean lists of characters in Spin/Spin2). Perhaps something like:

con
  NAME=string("hello, world")

would work though. It's a syntax error in both PNut and FlexSpin right now, so it shouldn't break any existing code.

Maybe these string constants passed down could also eventually be used for "include file" names in objects in order to patch in more custom values and code than the OBJ reference in the parent could realistically be expected to pass down? Although we don't have any include files in Chip's language version yet as I recall. Does flexspin already support include files by the way? EDIT: just checked the docs, yes it does.

The preprocessor (which handles #include) is, like the C preprocessor, strictly a text based macro processor which runs before any language parsing, so it can't meaningfully be modified by language features like CON and OBJ. However, it is pretty common to set up a set of constants in an object, like:

obj
   cfg : "config.spin2"

pub main()
  repeat
      pintoggle(cfg.LEDPIN)
      waitms(500)

(In old Spin you had to write cfg#LEDPIN, but I think PNut can use the dot syntax; certainly FlexSpin can.)

rogloh

@ersmith said:
The preprocessor (which handles #include) is, like the C preprocessor, strictly a text based macro processor which runs before any language parsing so it can't meaningfully be modified by language features like CON and OBJ. However, it is pretty common to set up a set of constants in an object, like:

Yeah it's okay when the constants are shared by all classes but if you wanted to customize the included files or customize the child classes' own included objects/methods from parameters in the parent it is not simple to achieve. Trying to find a common solution that works for both toolchains has failed so far but I guess it requires Chip's buy in to extend official SPIN2 language/capabilities. Maybe in future my drivers will just have to be for flex only if the pain continues.

evanh

This is primarily intended for binary code space saving, right? To build the binary with only the components needed for a specific request. Allowing more options like which external memory type is attached while not chewing up extra cogRAM to support the options.

rogloh

@evanh said:
This is primarily intended for binary code space saving, right? To build the binary with only the components needed for a specific request. Allowing more options like which external memory type is attached while not chewing up extra cogRAM to support the options.

Yeah, that's right. It would be nice to bring in code and API methods for only the types of memory that you require - not some superset of everything. For flexspin with dead code removal it's not so much of a big deal but it's not nice to manage everything in one giant monolithic file. I could try to put in #ifdefs everywhere but proptool still doesn't have that capability either, does it. It'd be nice to have the optional inclusion of child objects for what you want to support done at a file level, ideally controlled by a simple parameter passed down from the parent. Full object oriented code with inheritance etc could probably do what I want but we don't have that, and I don't really want all that baggage either to tell the truth. Just something that can customize what sub-objects or methods of my driver get included would probably suffice. The other main issue if you select different sub objects to include in a driver is that you need to go through an intermediate layer in order to wrap the various APIs beneath in a common way for the parent to call, which can be a performance hit for something like individual memory accesses. This is why including different files of driver source code methods can be useful.

pik33

This is OK:

class part
  dim part$ as string
  dim part_type as integer
  dim priority as integer
end class

dim lpart(125) as part

lpart(1).part$="test"
lpart(2).part$="test1"
print lpart(1).part$
print lpart(2).part$

This is also OK:

dim lpart as string(125)

lpart(1)="test"
lpart(2)="test1"
print lpart(1)
print lpart(2) 

but this is not:

class part
  dim part$ as string
  dim part_type as integer
  dim priority as integer
end class

dim lpart as part(125)

lpart(1).part$="test"
lpart(2).part$="test1"
print lpart(1).part$
print lpart(2).part$

The result:

"C:/Users/Piotr/Downloads/flexprop-6.1.5/flexprop/bin/flexspin" -2 -l --tabs=8 -D_BAUD=2000000   -O1    --charset=utf8 -I "C:/Users/Piotr/Downloads/flexprop-6.1.4/flexprop/include"  "D:/Programowanie/P2-retromachine-1/Propeller/Basic/test1.bas" -DFF_USE_LFN
Propeller Spin/PASM Compiler 'FlexSpin' (c) 2011-2023 Total Spectrum Software Inc. and contributors
Version 6.1.6-beta-v6.1.5-18-gc3609eec Compiled on: Jun  4 2023
test1.bas
D:/Programowanie/P2-retromachine-1/Propeller/Basic/test1.bas:2: error: Internal error, Taking size of an object with pending variables

D:/Programowanie/P2-retromachine-1/Propeller/Basic/test1.bas:2: error: Internal error, Taking size of an object with pending variables

child process exited abnormally
Finished at Mon Jun  5 13:08:27 2023

What happened here?

The real problem is: I cannot declare

type parts as part(125)

because it also gives this

error: Internal error, Taking size of an object with pending variables

ersmith

@pik33 : Hmm, that's an odd bug. I'll try to figure out what's going on, but for now you can work around it with

dim lpart(125) as part

ersmith

@pik33 : there was a missing check for arrays of objects when declaring variables. It should be fixed in github now.

ersmith

@pik33 said:
A feature request - "No terminal" option so the program loads and runs but opens no additional terminal window.

That's checked in to the "noterminal" branch in flexprop's github. If you want to try it out you don't actually need to compile anything, you can just copy src/gui.tcl into your flexprop/src directory (.tcl files are scripts, and gui.tcl contains most of the GUI code). The config file version number is bumped, so it shouldn't interfere with your existing flexprop config.

ersmith

@rogloh said:

@evanh said:
This is primarily intended for binary code space saving, right? To build the binary with only the components needed for a specific request. Allowing more options like which external memory type is attached while not chewing up extra cogRAM to support the options.

Yeah, that's right. It would be nice to bring in code and API methods for only the types of memory that you require - not some superset of everything. For flexspin with dead code removal it's not so much of a big deal but it's not nice to manage everything in one giant monolithic file. I could try to put in #ifdefs everywhere but proptool still doesn't have that capability either, does it. It'd be nice to have the optional inclusion of child objects for what you want to support done at a file level, ideally controlled by a simple parameter passed down from the parent. Full object oriented code with inheritance etc could probably do what I want but we don't have that, and I don't really want all that baggage either to tell the truth. Just something that can customize what sub-objects or methods of my driver get included would probably suffice. The other main issue if you select different sub objects to include in a driver is that you need to go through an intermediate layer in order to wrap the various APIs beneath in a common way for the parent to call, which can be a performance hit for something like individual memory accesses. This is why including different files of driver source code methods can be useful.

I'm goint to do some experimenting on this, but i think that (a) the best way forward may be to create some kind of interface declarations for objects, and (b) it's probably better to do the experimenting in BASIC, and once the feature is working to find a Spin syntax for it. I've started a thread to discuss this in the BASIC forum.

pik33

@ersmith said:

@pik33 said:
A feature request - "No terminal" option so the program loads and runs but opens no additional terminal window.

That's checked in to the "noterminal" branch in flexprop's github. If you want to try it out you don't actually need to compile anything, you can just copy src/gui.tcl into your flexprop/src directory (.tcl files are scripts, and gui.tcl contains most of the GUI code). The config file version number is bumped, so it shouldn't interfere with your existing flexprop config.

@ersmith said:
@pik33 : there was a missing check for arrays of objects when declaring variables. It should be fixed in github now.

Much better now. I needed a type, so I can pass a parameter of this type to a function. Of course I always could pass the pointer, but now it simply works without needing a workaround.

Also, as I am using a HDMI driver and I have Basic's file channel #0 captured by it, all prints go to this driver, so the terminal is empty now and only disturbs: there is still a bug that prevents the Flexprop to upload the program when the terminal is running. You have to close the terminal first or click "compile and run" twice.

pedward

FYI, I submitted a PR to add RPM build support for flexprop: https://github.com/totalspectrum/flexprop/pull/72

ersmith

Thank you @pedward . The RPM build support will be in the 6.2.0 release.

ersmith

I've posted version 6.1.7 of FlexProp on github and on Patreon. Changes include:

Version 6.1.7
- Added `DEBUG_DISABLE` constant to disable DEBUG() in Spin2 objects
- Added %b flag for binary output in printf
- Added ubin support for printf debug
- Made `virtual` a reserved keyword in C++
- Reduced memory usage of default printf

Version 6.1.6
- Added ability to double quote to include quotes in literal BASIC strings
- Allowed BASIC string constants to initialize other consts
- Fixed a problem with declaring new types that are arrays of objects
- Fixed a bug with calculating bits in BASIC direction() and output()
- Implemented flush() for FATFS files (thanks to Ada)
- Improved some error messages
- In C++ output, treat unknown types in prints as generic
- Made `interface` and `implements` reserved keywords in BASIC
- Made BASIC use of + operator on strings more consistent
- Optimized a few BASIC string expressions
- Optimized calculating length of string literals