FlexProp: a complete programming system for P2 (and P1)

evanh · 2023-06-19 17:04

Here's the compiled pasm using -O1 --fcache=0. It has inlined the whole call path. I suspect maybe the TESTP needs a spacer in front of it ...

'     ser.str(string("Press any key to skip count down : ")) 'tell user to press a key to avoid waiting
    mov local02, ##@LR__0041
'   REPEAT WHILE ((c := byte[s++]) <> 0)
LR__0011
    rdbyte  arg01, local02 wz
    add local02, #1
 if_e   jmp #LR__0013
'     tx(c)
'   wypin(tx_pin, val)
    wypin   arg01, #62
'   txflush() 'rja
'   repeat
LR__0012
'     z := pinr(tx_pin)
    testp   #62 wc
 if_ae  jmp #LR__0012
    jmp #LR__0011
LR__0013

Rayman · 2023-06-19 17:15

@ersmith Is there any way to get the internal PST compatible terminal from the command line?

evanh · 2023-06-19 17:16

@evanh said:
I suspect maybe the TESTP needs a spacer in front of it ...

Yeah, with the constants, the code has optimised down so much that an existing hidden bug in smartpin handling has emerged.

Rayman,
Something like this is a workaround:

PUB tx(val)    ' send one byte
  wypin(tx_pin, val)
  pinr(tx_pin)
  txflush() 'rja

EDIT: I guess in fairness, it's probably not the compiler's job to know the pipeline effects on a smartpin.

Wuerfel_21 · 2023-06-19 17:25

@Rayman said:
Is there any way to get the internal PST compatible terminal from the command line?

loadp2 -T -b <baud rate here> <the file you want to load (or not)> (notice the uppercase T)

Rayman · 2023-06-19 17:32

@Wuerfel_21 Thanks, that does it.

pik33 · 2023-06-19 18:07

This is a txflush that works with -O1

PUB tx(val)    ' send one byte
  wypin(tx_pin, val)
  txflush(tx_pin) 'rja

PUB txflush(txpin)  | z    ' wait for character sent

    asm
p1  rdpin z,txpin wc
    if_c jmp #p1
    endasm

Edit: I read the listing. Now this txflush is done in the fcache. That introduces a delay for fcache loading and it can be this delay that makes it work.

evanh · 2023-06-20 00:11

Pik,
That one will work anyway since RDPIN uses the smartpin's specialised bus rather than the IN bit via TESTP. However they are functionally a little different. RDPIN WC looks at shifter [not] empty flag whereas TESTP WC looks at buffer [not] full flag. This affects tx performance. TESTP is more desirable.

There is an input staging register on INA/INB signals between the smartpins and the cogs (Just like there is between the custom pad-ring and the smartpins). This is likely all it took to create our timing headache.

evanh · 2023-06-20 21:21

I've used a slightly different approach in my own routines when I wrote similar handlers before Spin2 was written. I chose the greatest buffering method, where the execution path is stalled only when there is a new character to be placed in the buffer but the buffer is still full. It's a little more complex than the exit-when-buffer-is-not-full approach above.

putch
'  input:  PB
' result:  (none)
'scratch:  (none)
        rqpin   inb, #DIAGTXPIN  wc 'transmiting? (C high == yes)  *Needed to initiate tx
        testp   #DIAGTXPIN  wz      'buffer free? (IN high == yes)
if_nc_or_z  wypin   pb, #DIAGTXPIN      'write new byte to Y buffer
if_nc_or_z  ret wcz         'restore C/Z flags of calling routine
        jmp #putch          'wait while Smartpin is both full (nz) and transmitting (c)

A beneficial side effect of its structure, which I didn't appreciate, is it has plenty of time from WYPIN to TESTP.

pik33 · 2023-06-21 11:44

The next glitch detected. This is still the same project:

https://gitlab.com/pik33/P2-retromachine/-/tree/main/Propeller/Basic - the file is https://gitlab.com/pik33/P2-retromachine/-/blob/main/Propeller/Basic/basic003.bas

function expr(ct as integer) as expr_result,integer

' On input: ct = current token position
' On output: expression result value and a new ct

dim t1, t2 as expr_result
dim op as integer

t1,ct = muldiv(ct)                      ' call higher priority operator check. It will itself call getval/getvar if no multiplies or divides
op = lparts(ct).token               ' that idea is from github adamdunkels/ubasic

do while (op = token_plus orelse op = token_minus orelse op = token_and orelse op=token_or)
  ct+=1
  t2,ct = muldiv(ct) 
  select case op
    case token_plus
      t1=do_plus(t1,t2)
    case token_minus
      t1=do_minus(t1,t2)
    case token_and
      t1=do_and(t1,t2)
    case token_or
      t1=do_or(t1,t2)
  end select  
  op = lparts(ct).token
  loop
return t1,ct
end function

There are 3 other functions that are declared as fun(integer) as expr_result, integer. And the result is

"C:/Users/Piotr/Downloads/flexprop-6.1.5/flexprop/bin/flexspin" -2  --tabs=8 -D_BAUD=230400 -O1    --charset=utf8 -I "C:/Users/Piotr/Downloads/flexprop-6.1.5/flexprop/include"  "D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas" -DFF_USE_LFN -DFF_CODE_PAGE=852
Propeller Spin/PASM Compiler 'FlexSpin' (c) 2011-2023 Total Spectrum Software Inc. and contributors
Version 6.1.9-beta-v6.1.8-14-g3fac69b7 Compiled on: Jun 18 2023
basic003.bas
|-hg009.spin2
|-|-psram.spin2
|-|-|-psram16drv.spin2
|-psram.spin2
|-usbnew.spin2
|-audio093b-8-sc.spin2
D:/Programowanie/P2-retromachine-1/Propeller/Basic/hg009.spin2:363: warning: used spaces for indentation (previous lines used tabs)
D:/Programowanie/P2-retromachine-1/Propeller/Basic/hg009.spin2:955: warning: used spaces for indentation (previous lines used tabs)
D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:536: error: inconsistent return values from function expr: expected 4 elements but found 2
D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:568: error: inconsistent return values from function muldiv: expected 4 elements but found 2
D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:592: error: inconsistent return values from function getvalue: expected 4 elements but found 2
D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:604: error: inconsistent return values from function getvar: expected 4 elements but found 2
child process exited abnormally
Finished at Wed Jun 21 13:35:43 2023

All these lines where the error was detected are exatly the same: return t1,ct

This is the result from the newest github commit; the previous version I used (6.1.6) crashed with a segmentation error on this code.

pik33 · 2023-06-21 13:20

Another problem. The simplified test code:

union eresult
  dim iresult as integer
  dim uresult as ulong
  dim lresult as longint
  dim ulresult as ulongint
  dim fresult as double
  dim sresult as string
  end union

class expr_result
dim result as eresult
dim result_type as ubyte  
end class

class part
  dim part$ as string
  dim token as integer
  dim priority as integer
end class

type parts as part(125) 
dim lparts as parts
dim ct as integer
dim t1 as expr_result

const token_decimal=512

'---------------------------------------
print "Expected result=2, result type=1"
print

lparts(0).part$="2"
lparts(0).token=token_decimal
ct=0

t1=expr()
print "Expression result="; t1.result.uresult
print "Result type=";t1.result_type

'-------------------------------------

function expr() as expr_result

dim t1 as expr_result

t1 = muldiv()   
print "in expr: result=";t1.result.uresult
print "in expr: result type=";t1.result_type
print
return t1
end function


function muldiv() as expr_result

dim t1 as expr_result

t1 = getvalue()    
print "in muldiv: result=";t1.result.uresult
print "in muldiv: result type=";t1.result_type
print
return t1    
end function

function getvalue() as expr_result

dim t1 as expr_result
dim op as integer

op=lparts(ct).token

select case op
  case token_decimal
    t1.result.uresult=val%(lparts(ct).part$): t1.result_type=1  
    print "in getvalue: result=";t1.result.uresult
    print "in getvalue: result type=";t1.result_type
    print
  end select  
return t1
end function

The result:

( Entering terminal mode.  Press Ctrl-] or Ctrl-Z to exit. )                    
Expected result=2, result type=1                                                

in getvalue: result=2                                                           
in getvalue: result type=1                                                      

in muldiv: result=2147489292                                                    
in muldiv: result type=10                                                       

in expr: result=4127513600                                                      
in expr: result type=76                                                         

Expression result=0                                                             
Result type=0

Edit: tried using different var names are different in every function, the result is the same. Compiled with Version 6.1.9-beta-v6.1.8-14-g3fac69b7 Compiled on: Jun 18 2023

pik33 · 2023-06-22 08:10

The latest commit now gives (a bunch of) these errors:

D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:518: error: Unable to multiply assign this target
D:/Programowanie/P2-retromachine-1/Propeller/Basic/basic003.bas:518: error: Too many elements on right hand side of assignment

in lines that call the functions like this:

t1,ct=expr(ct)

The test program from post #1451 still produces improper results

This is even more simplified test code. No complex class, no nested class, no unions.

class expr_result
  dim uresult as integer
  dim result_type as ubyte  
end class

dim t1 as expr_result
'---------------------------------------
print "Expected result=2, result type=1"
print

t1=expr()

print "Expression result="; t1.uresult
print "Result type=";t1.result_type

'-------------------------------------

function expr() as expr_result

dim t2 as expr_result

t2 = muldiv()   
print "in expr: result=";t2.uresult
print "in expr: result type=";t2.result_type
print
return t2
end function


function muldiv() as expr_result

dim t3 as expr_result

t3 = getvalue()    
print "in muldiv: result=";t3.uresult
print "in muldiv: result type=";t3.result_type
print
return t3    
end function

function getvalue() as expr_result

dim t4 as expr_result
dim op as integer

t4.uresult=2: t4.result_type=1 ' todo token_int64
print "in getvalue: result=";t4.uresult
print "in getvalue: result type=";t4.result_type
print

return t4
end function

( Entering terminal mode.  Press Ctrl-] or Ctrl-Z to exit. )                    
Expected result=2, result type=1                                                

in getvalue: result=2                                                           
in getvalue: result type=1                                                      

in muldiv: result=10                                                            
in muldiv: result type=136                                                      

in expr: result=-167718803                                                      
in expr: result type=136                                                        

Expression result=0                                                             
Result type=0

pik33 · 2023-06-22 08:22

In the test code above, if t2, t3, t4 are declared global at the start of the program, the result is:

( Entering terminal mode.  Press Ctrl-] or Ctrl-Z to exit. )                    
Expected result=2, result type=1                                                

in getvalue: result=2                                                           
in getvalue: result type=1                                                      

in muldiv: result=0                                                             
in muldiv: result type=0                                                        

in expr: result=0                                                               
in expr: result type=0                                                          

Expression result=0                                                             
Result type=0

The more interesting thing happens if

class expr_result
  dim uresult as integer
  dim result_type as ubyte  
end class

dim t1,t2,t3,t4 as expr_result
'---------------------------------------
print "Expected result=2, result type=1"
print

t1=expr()

print "Expression result="; t1.uresult
print "Result type=";t1.result_type

'-------------------------------------

function expr() as expr_result

'dim t2 as expr_result

t2 = muldiv()   
print "in expr: result=";t2.uresult
print "in expr: result type=";t2.result_type
print
return t2
end function


function muldiv() as expr_result

'dim t3 as expr_result

t3 = getvalue()    
print "in muldiv: result=";t3.uresult
print "in muldiv: result type=";t3.result_type
print
return t3    
end function

function getvalue() as expr_result

'dim t4 as expr_result


t4.uresult=2: t4.result_type=1 ' todo token_int64
print "in getvalue: result=";t4.uresult
print "in getvalue: result type=";t4.result_type
print
t3=t4
t4.uresult=12345
return t4
end function

The result is:

( Entering terminal mode.  Press Ctrl-] or Ctrl-Z to exit. )                    
Expected result=2, result type=1                                                

in getvalue: result=2                                                           
in getvalue: result type=1                                                      

in muldiv: result=2                                                             
in muldiv: result type=1                                                        

in expr: result=0                                                               
in expr: result type=0                                                          

Expression result=0                                                             
Result type=0

pik33 · 2023-06-22 08:45

The simplest code:

class expr_result
  dim uresult as integer
  dim result_type as ubyte  
end class

dim t3,t4 as expr_result
'---------------------------------------
print muldiv().uresult
'-------------------------------------

function muldiv() as expr_result

'dim t3 as expr_result

t3.uresult=23456
print "Returned uresult: ";getvalue().uresult
t3 = getvalue() 
print "Assigned uresult:";t3.uresult
print
return t3    
end function

function getvalue() as expr_result

t4.uresult=2
return t4
end function

gives

Returned uresult: 2                                                             
Assigned uresult:23456                                                          

23456

However I can assign a single field and it works

class expr_result
  dim uresult as integer
  dim result_type as ubyte  
end class

dim t3,t4 as expr_result
'---------------------------------------
print muldiv().uresult
'-------------------------------------

function muldiv() as expr_result

'dim t3 as expr_result

t3.uresult=23456
print "Returned uresult: ";getvalue().uresult
t3.uresult = getvalue().uresult 
print "Assigned uresult:";t3.uresult
print
return t3    
end function

function getvalue() as expr_result

t4.uresult=2
return t4
end function

Returned uresult: 2                                                             
Assigned uresult:2                                                              

2

evanh · 2023-06-22 09:50

Oops, about the serial TX flags, I got their function a little wrong in my description above. IN (TESTP) is a buffer empty rather than a buffer full flag. The difference suits events and interrupts which have data initiation from another routine so as to transition the TX buffer handling from idle to active, and naturally falls back to idle once data is transmitted.

So that's why my handler needs the extra RQPIN WC to get it going without using events. The extra complexity wasn't an oversight in smartpin wiring.

ersmith · 2023-06-22 13:20

@pik33 : There are a whole bunch of bugs in function return and multiple assignment . For now I'd offer the following advice to avoid these bugs:

(1) Don't try to use any structures or unions in a multiple assignment (that is, in a, b = foo(b) neither a nor b should be a class). Use BYREF function parameters instead.

(2) Try to use only 4 byte or 8 byte sized items in classes that will be returned from functions: no ubyte or ushort in classes that will be returned from functions.

I hope to fix at least some of the bugs soon, but there are several intersecting issues so it's complicated.

pik33 · 2023-06-22 14:09

When changed ubyte to ulong, the last simple test code works.

Edit: more than 3 elements in the class, even if they are 4 bytes, fails.

ersmith · 2023-07-13 02:19

FlexProp 6.2.1 is available now. This has a revamped system for method pointers, support for having multiple littlefs file systems (including both flash and RAM disks), a new BASIC interface class system, and the ability to run with no window.

Note that the config file format has changed, so the FlexProp 6.2.1 config (things like the baud rate, optimization options, and so on) will be reset to the default when you first start it up. Older FlexProp config files can co-exist with the new one (they have different names).

Mickster · 2023-07-13 09:27

@ersmith

Hello Eric

P2 Edge Rev B

rem simple program to toggle a pin
const pin = 56
direction(pin) = output
do
output(pin) = not output(pin) 'With both FB 6.1.2 and now 6.2.1, The LED goes off and never changes
'pintoggle(pin)       'I replace the above with pintoggle and I get a flashing LED
pausems 1000
loop

Loving littlefs, BTW

Mickster · 2023-07-13 10:01

Same problem with:

rem simple program to toggle a pin
const pin = 56
dim state
state = 0
direction(pin) = output
do
pausems 500
state = not state
output(pin)=state
pausems 500
loop

Craig

ersmith · 2023-07-13 12:02

Thanks @Mickster , there was a problem with constant pins above 32 not working correctly because of some 64 bit constant changes; that's actually been around for a while. It'll be fixed in the next release.

Mickster · 2023-07-13 12:51

@ersmith said:
Thanks @Mickster , there was a problem with constant pins above 32 not working correctly because of some 64 bit constant changes; that's actually been around for a while. It'll be fixed in the next release.

Right-o
Bit intrigued by the mention of "RAM disk" Couldn't find any details in the Basic.pdf

(including both flash and RAM disks)

Edit: Guessing that "RAM" is the onboard flash and the "flash" is the SD card.

Rgds,

Craig

pik33 · 2023-07-13 14:53

RAM is a HUB RAM, flash is a (part of a) flash These use littlefs. SD card is SD card, it uses FAT32. A PSRAM can make a good ramdisk... but this is yet to be written...

Maybe we need something like in 8-bit Atari OS: you can define a block device, write read block and write block procedure, and then mount it. We have already a character devices implemented in FlexBasic (SendRecvDevice), so maybe it can be also done for block devices? Something like this:

mount "/dev", _vfs_open_blockdevice(init,readblock,writeblock,close,fstype)

ersmith · 2023-07-13 15:45

@Mickster said:

show previous quotes

@ersmith said:
Thanks @Mickster , there was a problem with constant pins above 32 not working correctly because of some 64 bit constant changes; that's actually been around for a while. It'll be fixed in the next release.

Right-o
Bit intrigued by the mention of "RAM disk" Couldn't find any details in the Basic.pdf

show previous quotes

(including both flash and RAM disks)

Edit: Guessing that "RAM" is the onboard flash and the "flash" is the SD card.

RAM is either HUB RAM or an external add-on RAM (like hyperram). There's an example in the samples/shell/shell.c file that comes with FlexProp.

Mickster · 2023-07-13 19:51

Just started to play with double precision and expecting clock cycles to be gobbled-up.... Doesn't happen is this Ada or Eric wizardry?

Craig

pik33 · 2023-07-13 19:53

There is no double precision floats yet. You can declare and use them, but the computation is done in single resolution.

However, 64bit integers already work.

ersmith · 2023-07-13 21:16

@pik33 said:
RAM is a HUB RAM, flash is a (part of a) flash These use littlefs. SD card is SD card, it uses FAT32. A PSRAM can make a good ramdisk... but this is yet to be written...

I took some PSRAM code from @Wuerfel_21 's MegaYume project, but it doesn't seem to be working, no doubt due to my screwing something up. It's in include/spin/psram*.spin2. @rogloh 's HyperRam/HyperFlash driver does work though for the ramdisk. The existing LittleFS config structure is used to set up alternate drivers; that is, you can call _vfs_open_littlefs_flash(fmt, config) where fmt is a flag (0 = do not format the storage) and config is either 0 (to use the default flash driver) or a block of data as follows:

config(0) = flash page size in bytes (typically 256)
config(1) = erase page size in bytes (typically 4K or 64K; use the config(0) value for RAM disks)
config(2) = starting offset within flash, must be a multiple of config(1)
config(3) = size of area to use, must be a multiple of config(1)
config(4) = pointer to block device info, or 0 for default device; see below
config(5) = pin mask low; mask indicating which pins the driver will use
config(6) = pin mask high; mask indicating which pins the driver will use
config(7) = reserved, set to 0

The block device structure has 7 words and looks like:

class block_device
  dim blk_read as function(dst as any ptr, flashAddr as uinteger, size as uinteger) as integer
  dim blk_write as function(src as any ptr, flashAdr as uinteger) as integer  ' write 1 block
  dim blk_erase as function(flashAdr as uinteger) as integer                  ' erase 1 block
  dim blk_sync as function() as integer
  dim read_cache as ubyte ptr  ' points to a config(0) sized scratch area
  dim write_cache as ubyte ptr ' points to another config(0) sized scratch area
  dim look_cache as ubyte ptr  ' points to a third config(0) sized scratch area
end class

Mickster · 2023-07-13 21:37

@pik33 said:
There is no double precision floats yet. You can declare and use them, but the computation is done in single resolution.

However, 64bit integers already work.

Boy am I red faced 🤣😂

I guess the good news is that I don't really need them because I get more precision than I can use

Already using 64bit integers to extend my encoder counter. I guess it might roll over in a few thousand years but hey, I'll worry about it (much) later 😂🤣

Craig

Wuerfel_21 · 2023-07-13 21:50

@ersmith said:
I took some PSRAM code from @Wuerfel_21 's MegaYume project, but it doesn't seem to be working, no doubt due to my screwing something up. It's in include/spin/psram*.spin2. @rogloh 's HyperRam/HyperFlash driver does work though for the ramdisk. The existing LittleFS config structure is used to set up alternate drivers; that is, you can call _vfs_open_littlefs_flash(fmt, config) where fmt is a flag (0 = do not format the storage) and config is either 0 (to use the default flash driver) or a block of data as follows:

Yea, I saw you did something there... Main takeaway is that the delay values used for roger's driver setup aren't actually correct in the emulators because I gave up on trying to keep them synced with the custom access code. Also, they need to be tuned based on clkfreq. There's a tester program out there somewhere that helps identify the ideal range. Speaking of custom access code, for a ramdisk in the ususal sort of synchronous C file I/O framework, just make some simplified read/write functions and run them in fcache instead of wasting a cog on it. If the application brings it's own cog mode RAM driver for advanced usage it can hook up the ramdisk functions itself. Though I haven't really written RAM init code yet, but that shouldn't be too hard, either.

ersmith · 2023-07-19 18:34

I've uploaded binary releases of FlexProp 6.2.3 to both github and my Patreon page. The changes from 6.2.1 are:

Version 6.2.3
- Added strpbrk(), strcoll(), strspn() functions for the C library
- Fixed a potential crash in Spin2 parsing of @func()
- Fixed taking the difference of a pointer and generic value

Version 6.2.2
- Fixed some 64 bit constant issues with DIR/OUT/IN manipulation
- Make sure garbage collector does not reclaim BASIC I/O wrapper
- Reverted waitatn() change from 6.2.0

Rayman · 2023-07-19 23:26

@ersmith A PSRAM based file system would be pretty neat. Lot of moving parts to make that work I think. Not exactly sure how practical for use it is, but interesting to think about.
I'd personally target the 16-bit interface like that Parallax Edge board has. That would have a speed advantage over flash/uSD I think.

The littlefs for extra flash space though is really nice. I do want to try that out soon.

FlexProp: a complete programming system for P2 (and P1)

Comments