New BASIC compiler for Prop1 and Prop2 — Parallax Forums

# New BASIC compiler for Prop1 and Prop2

Posts: 5,399
edited 2019-08-14 16:14
Edit August 2019: It's been almost a year since I started the project, and I think BASIC support in fastspin is now very mature. We have some solid features like:
- Broadly FreeBASIC / MS BASIC compatible (including support for really old programs that use line numbers and gosub
- A simple preprocessor that allows #define, #ifdef / #else / #endif, for conditional compliation and simple macro substitution
- Inline assembly inside functions and subroutines, or in the main program
- Support for floats, strings, integers, pointers, arrays, and user defined structures
- Produce optimized Propeller executables for both P1 and P2
- The same compiler supports PASM, Spin, BASIC, and C, so functions written in any of those languages can call each other.

The early part of this thread has got some thrashing around about the language design, you can ignore that (most of it is obsolete). In the end I decided to make the strings be garbage collected, and this vastly simplified things. File I/O is done with traditional BASIC "open" and "print #n, x" style statements.

I've attached the current PDF documentation to this message so you can see what the language looks like and what features it has.

Rather than trying to keep the originally attached .zip here up to date, I'll just add pointers. Note that "spin2gui" can be used for BASIC development as well as Spin, and works for both P1 and P2:

spin2gui: https://github.com/totalspectrum/spin2gui/releases
fastspin: https://github.com/totalspectrum/spin2cpp/releases

spin2gui contains fastspin, proploader, loadp2, and a simple editor, so it has everything you need to try things out on Windows. Linux/Mac users can build fastspin and spin2gui themselves from the source code. I develop on Linux, so that should definitely work, and I think there are some Mac users as well.

Edit: The work in progress compiler is attached here. To call it, use a command line:
  fastspin myprog.bas

which will produce myprog.binary. I've left the rest of the message the same, but some of it is obsolete... see the thread for discussions on how the language has evolved.

I'm working on a BASIC compiler for Prop2 (which will incidentally support Prop1 too, since it's based on fastspin which handles both). It'll be similar to PropBasic in that it will compile to COG or LMM code, but it's not a PropBasic replacement -- the intention is to make a more Microsoft like syntax, rather than PropBasic's pbasic syntax.

What features would you like to see in a BASIC compiler for Prop1 (and/or Prop2)? I've got the following things planned:

(1) Support for using Spin objects (so easy access to existing objects)
(2) Floating point and string support built in. Types are either inferred from the name ("a$" is a string, "a" is an integer) or explicitly declared in a DIM statement. (3) Syntax that's a subset of FreeBasic. (4) Optimized PASM code output (5) Can directly build binaries (no need for bstc or any other Spin compiler) The string support is probably going to be the hardest part, since BASIC traditionally has pretty powerful string handling, much more so than Spin or C. At present I'm thinking of limiting strings to 255 characters in length to simplify some of the code. Is that too restrictive? How important are multi-dimensional arrays? At present fastspin just supports one dimensionsal arrays, so there's some work to do to add multiple dimensions, but it is a traditional BASIC feature. It's still quite a ways away from production, although the curious can get it from the spin2cpp GitHub repository (you'll have to build it yourself from the "basic" branch; again, it's nowhere near ready for production, so I won't be releasing binaries for a while yet). Today I got the first programs running. Here's a sample of something that works: '' '' import the Spin FullDuplexSerial object '' class fullduplex using "FullDuplexSerial.spin" '' create a full duplex serial object dim ser as fullduplex '' start the serial ser.start(31, 30, 0, 115_200) rem note that as usual in basic, the variable i is rem declared automatically for i = 1 to 10 ser.dec(i) newline() next i do rem loop forever loop sub newline() ser.tx(13) ser.tx(10) end sub  ## Comments • Posts: 14,475 Cool! Are you going to have a heap for dynamic strings? • Posts: 14,950 edited 2018-09-03 23:19 ersmith wrote: » I'm working on a BASIC compiler for Prop2 (which will incidentally support Prop1 too, since it's based on fastspin which handles both). It'll be similar to PropBasic in that it will compile to COG or LMM code, but it's not a PropBasic replacement -- the intention is to make a more Microsoft like syntax, rather than PropBasic's pbasic syntax. What features would you like to see in a BASIC compiler for Prop1 (and/or Prop2)? I've got the following things planned: (1) Support for using Spin objects (so easy access to existing objects) (2) Floating point and string support built in. Types are either inferred from the name ("a$" is a string, "a" is an integer) or explicitly declared in a DIM statement.
(3) Syntax that's a subset of FreeBasic.
(4) Optimized PASM code output
(5) Can directly build binaries (no need for bstc or any other Spin compiler)

Sounds good, I do like the 'more FreeBasic compatible' idea.

I recently added a patch to PropBasic to allow PoseidonFB IDE (see below image) to call PropBASIC, and pick up the error messages for error-line-highlight.
Was quite easy to do, just a minor shuffle of the error report line, to be FreeBASIC cloned.
That allows you to quickly leverage the various FreeBASIC IDEs - I was using FBide, but PoseidonFB seems to be gaining traction. There is also FBedit.

Will it include the same conditional preprocessor FreeBASIC does, so you can have one source for both - allows PCs to do testing of functions / code.

What about Asm..End Asm to allow in-line assembler ie same syntax as FreeBASIC ?

Could it give a slightly better 'not supported' type message on places where FreeBASIC is a superset ? (rather than a syntax error)

ersmith wrote: »
The string support is probably going to be the hardest part, since BASIC traditionally has pretty powerful string handling, much more so than Spin or C. At present I'm thinking of limiting strings to 255 characters in length to simplify some of the code. Is that too restrictive?
Hmm, on small MCUs, 255 is probably tolerable, so for a P1 that's likely ok. However, P2 is not so small anymore... - could this be some system option ?
- eg smaller strings for more compact code (eg P1), but larger ones allowed for P2 ?

ersmith wrote: »
How important are multi-dimensional arrays? At present fastspin just supports one dimensionsal arrays, so there's some work to do to add multiple dimensions, but it is a traditional BASIC feature.
Yes, that's nice to have, but does not need to be in the first release.

Modified PropBASIC called from PoseidonFB IDE, with error parsing shown : (/FB command line option now included in latest PropBASIC release, to reformat error reports)
• Posts: 5,399
David Betz wrote: »
Cool! Are you going to have a heap for dynamic strings?

Well, dynamic strings would pretty much require garbage collection, which seems like a lot of trouble. I was hoping to get away with putting the maximum length of the string in the upper bits of the pointer. Then we'd be able to translate code like "A$= B$ + C$" into something like:  alen = ((unsigned)a) >> 24; blen = ((unsigned)b) >> 24; strncpy(a, b, min(alen, blen)); clen = ((unsigned)c) >> 24; strncat(a, c, min(alen, clen));  On the P2 we'd actually have 12 bits for the length, so strings could go up to 4095 long, but 255 is more "traditional" and also would allow for an alternate implementation where the first byte of the string array held the length. • Posts: 14,475 ersmith wrote: » David Betz wrote: » Cool! Are you going to have a heap for dynamic strings? Well, dynamic strings would pretty much require garbage collection, which seems like a lot of trouble. I was hoping to get away with putting the maximum length of the string in the upper bits of the pointer. Then we'd be able to translate code like "A$ = B$+ C$" into something like:
   alen = ((unsigned)a) >> 24;
blen = ((unsigned)b) >> 24;
strncpy(a, b, min(alen, blen));
clen = ((unsigned)c) >> 24;
strncat(a, c, min(alen, clen));

On the P2 we'd actually have 12 bits for the length, so strings could go up to 4095 long, but 255 is more "traditional" and also would allow for an alternate implementation where the first byte of the string array held the length.
But if you have more complicated string expressions with nested calls to string functions you will probably need a way to create temporary strings on the fly.

• Posts: 5,399
jmg wrote: »
I recently added a patch to PropBasic to allow PoseidonFB IDE (see below image) to call PropBASIC, and pick up the error messages for error-line-highlight.
Was quite easy to do, just a minor shuffle of the error report line, to be FreeBASIC cloned.
That sounds like a good idea. Do the IDEs have an easy way to change compiler and the way the output is run? It'd be nice to be able to call fastbasic (or whatever it ends up being called) and run the compiled binary directly from the IDE.
Will it include the same conditional preprocessor FreeBASIC does, so you can have one source for both - allows PCs to do testing of functions / code.

Initially it will have the same preprocessor as fastspin / openspin, which is pretty basic but does support simple #define, #ifdef / #else / #endif.
What about Asm..End Asm to allow in-line assembler ie same syntax as FreeBASIC ?
I'd like to do this. It's using the same engine as fastspin, so in principle supporting inline assembly is not a problem. Actually parsing the assembly will be a bit of a pain since in the Spin case I was able to re-use the DAT section parsing, whereas for BASIC I'll have to re-implement that. So maybe not the first release? We'll have to see.
Could it give a slightly better 'not supported' type message on places where FreeBASIC is a superset ? (rather than a syntax error)
Hmmm. That's also a good idea, and I think it could be done in at least some cases, but probably not for all of them.
• Posts: 5,399
David Betz wrote: »
ersmith wrote: »
David Betz wrote: »
Cool! Are you going to have a heap for dynamic strings?

Well, dynamic strings would pretty much require garbage collection, which seems like a lot of trouble. I was hoping to get away with putting the maximum length of the string in the upper bits of the pointer.
But if you have more complicated string expressions with nested calls to string functions you will probably need a way to create temporary strings on the fly.

True, but temporaries could be created on the stack, which will get freed when the function returns, so no need for garbage collection. Figuring out the size of temporaries could be a little tricky... I guess we could use the max of the lengths of any strings involved. Or maybe we just disallow any expressions that are too complicated to easily be converted? The only string returning operators I was planning to support initially were concatenation ("+") and the substring functions (LEFT$, MID$, RIGHT$). Hmmm, but for user defined functions I guess things get complicated. Dynamic strings would definitely make things easier on the compiler, but I worry about the size and space implications, not to mention the need for a garbage collector. • Posts: 14,475 ersmith wrote: » David Betz wrote: » ersmith wrote: » David Betz wrote: » Cool! Are you going to have a heap for dynamic strings? Well, dynamic strings would pretty much require garbage collection, which seems like a lot of trouble. I was hoping to get away with putting the maximum length of the string in the upper bits of the pointer. But if you have more complicated string expressions with nested calls to string functions you will probably need a way to create temporary strings on the fly. True, but temporaries could be created on the stack, which will get freed when the function returns, so no need for garbage collection. Figuring out the size of temporaries could be a little tricky... I guess we could use the max of the lengths of any strings involved. Or maybe we just disallow any expressions that are too complicated to easily be converted? The only string returning operators I was planning to support initially were concatenation ("+") and the substring functions (LEFT$, MID$, RIGHT$). Hmmm, but for user defined functions I guess things get complicated. Dynamic strings would definitely make things easier on the compiler, but I worry about the size and space implications, not to mention the need for a garbage collector.
At least a string garbage collector doesn't have to worry about recursively looking for references to heap objects. A string is just characters and can't refer to another object. It's simpler than general-purpose garbage collection.

• Posts: 5,399
David Betz wrote: »
At least a string garbage collector doesn't have to worry about recursively looking for references to heap objects. A string is just characters and can't refer to another object. It's simpler than general-purpose garbage collection.

But finding all the references to the strings requires looking through stack, heap, and registers, doesn't it?

Maybe reference counted strings would be the way to go. Since the string doesn't contain any pointers it can't have loops, so at least in theory it should be do-able. I still kind of like the static allocation idea though because then you know how much space your program will need. Maybe a compromise where most strings are statically allocated but you can define a size of a string heap for temporaries?

• Posts: 5,399
Another question I had for everyone was how to handle PRINT. On the Prop I guess we can default to printing at some fixed baud rate over the standard serial pins, but for many purposes people will want to change this. How do other MCU BASICs handle this?

One thought I had was allowing a PRINT WITH to specify a method to use for printing characters. The parameter would be an object and method that takes a single integer parameter and outputs it. Something like:
class fdserial input "FullDuplexSerial.spin"
class vga input "VGA.spin"

dim ser as fdserial
dim screen as vga

print with ser.tx
print "This message will go to serial"
print "So will this one"

print with vga.putchar
print "This message and all subsequent ones will go to VGA"

• Posts: 14,475
Some old versions of BASIC used an ugly syntax like this:
print #1, x


This would print to whatever device was opened as #1. This could be a file or I guess it could be a TV of VGA driver. I think you opened files like this:
open "foo.txt" as file 1

• Posts: 14,475
ersmith wrote: »
David Betz wrote: »
At least a string garbage collector doesn't have to worry about recursively looking for references to heap objects. A string is just characters and can't refer to another object. It's simpler than general-purpose garbage collection.

But finding all the references to the strings requires looking through stack, heap, and registers, doesn't it?

Maybe reference counted strings would be the way to go. Since the string doesn't contain any pointers it can't have loops, so at least in theory it should be do-able. I still kind of like the static allocation idea though because then you know how much space your program will need. Maybe a compromise where most strings are statically allocated but you can define a size of a string heap for temporaries?
Yup, you still have to find all string references. I realized that the really old versions of BASIC had it easy because they only really had global variables so you could always find all references.

• Posts: 23,342
I think the Propeller is supremely suited to doing garbage collection. After all, it can be accomplished in a separate cog, with minimal effect on determinism -- save some locking and unlocking as things get moved around in hub RAM.

-Phil
• Posts: 14,950
ersmith wrote: »
jmg wrote: »
I recently added a patch to PropBasic to allow PoseidonFB IDE (see below image) to call PropBASIC, and pick up the error messages for error-line-highlight.
Was quite easy to do, just a minor shuffle of the error report line, to be FreeBASIC cloned.
That sounds like a good idea. Do the IDEs have an easy way to change compiler and the way the output is run? It'd be nice to be able to call fastbasic (or whatever it ends up being called) and run the compiled binary directly from the IDE.

Yes, in the test example above, I simply swapped in the 'Compiler path' from the default ..fbc.exe to a path\to\CallPropBasic.BAT and that file could include a download if no errors line.
Because fbc.exe is a standalone compiler most IDEs should be able to manage this.
ersmith wrote: »
Will it include the same conditional preprocessor FreeBASIC does, so you can have one source for both - allows PCs to do testing of functions / code.

Initially it will have the same preprocessor as fastspin / openspin, which is pretty basic but does support simple #define, #ifdef / #else / #endif.

That sounds good enough
The FB help says
https://www.freebasic.net/wiki/wikka.php?wakka=CatPgDddefines

which gives examples of
#if __FB_VERSION__ < "0.18"
#error  Please compile With FB version 0.18 Or above
#endif

Print __FB_SIGNATURE__
yields
FreeBASIC 0.21.1


that means a #ifdef __FB_VERSION__ test should be able to switch between platforms. Shame fastbasic truncates to FB ?
ersmith wrote: »
What about Asm..End Asm to allow in-line assembler ie same syntax as FreeBASIC ?
I'd like to do this. It's using the same engine as fastspin, so in principle supporting inline assembly is not a problem. Actually parsing the assembly will be a bit of a pain since in the Spin case I was able to re-use the DAT section parsing, whereas for BASIC I'll have to re-implement that. So maybe not the first release? We'll have to see.
It would be nice to have, as it also allows users to learn PASM, without having to learn all of PASM. If the listings out have ASM/source included as commment, that also helps them learn & paste/modify.

• Posts: 14,950
ersmith wrote: »
Another question I had for everyone was how to handle PRINT. On the Prop I guess we can default to printing at some fixed baud rate over the standard serial pins, but for many purposes people will want to change this. How do other MCU BASICs handle this?
FreeBASIC codes like this
If Open Com ("COM16:115200,n,8,1,cs0,rs,ds0,cd0,bin,op2000,TB32000,RB32000" For Binary As #1) <> 0 Then  'generic baud
n = Err()
Print "unable to open serial port, (press any key) Error Code: ";n
Sleep
End
else
n = Err()
Print "No error on Open Com, Error Code: "; n  'always 0 ?
End If

Print "Sending command: AT+CrLf"
Print #1, "AT" + Chr(13, 10);


ersmith wrote: »
One thought I had was allowing a PRINT WITH to specify a method to use for printing characters. The parameter would be an object and method that takes a single integer parameter and outputs it. Something like:
class fdserial input "FullDuplexSerial.spin"
class vga input "VGA.spin"

dim ser as fdserial
dim screen as vga

print with ser.tx
print "This message will go to serial"
print "So will this one"

print with vga.putchar
print "This message and all subsequent ones will go to VGA"


That's easy to read, but print #n, may be easier to test on PC hosts for example.
eg I'm thinking here about using FB debuggers, to test Data/String/Flow code, complete with VAR watch etc, rather like a simulator, until the code is shaken down enough to try on a real chip.

In the above example, the serial print would either go to a PC serial port, or a Prop port, with a conditional variant of the Open Com line.
• Posts: 14,950
edited 2018-09-04 04:29
ersmith wrote: »
Another question I had for everyone was how to handle PRINT. On the Prop I guess we can default to printing at some fixed baud rate over the standard serial pins, but for many purposes people will want to change this. How do other MCU BASICs handle this?

Thinking about how someone might use PC COM ports to test, and also use Screen output for other messages, I ran some tests in FreeBasic (1.05).
These were revealing, as not all means of screen write are identical, but they do show you could flip between COM ports and Cons, Scrn targets for useful debug & development.
"CON" behaves slightly differently.

FreeBASIC test code and results captured.
REM Blank line added
Dim a as Integer
Dim ern as Integer
Dim s as String
Dim L as Long

print "1 hello out there, Started JTest"

open Cons for output as #1    ' Works
open "CON" for output as #2   ' Works but delays until close
open Scrn for output as #3    ' Works
ern=err
print "err=",ern
#define TestPrint
#IFDEF TestPrint
print #1," 2 Opened Cons as #1 ";
print #2," 3 print #2 statement";      ' No new line
print #3," 4 print #3 statement";      ' No new line
sleep 2000
print #1," 5 second half Cons ";        ' No new line
print " 6 hello out there "
#ELSE  'Test put
put #1, ," 2 Opened Cons as #1 "
put #2, ," 3 put #2 statement"         ' No new line
put #3, ," 4 put #3 statement"         ' No new line
sleep 2000
put #1, ," 5 second half Cons "         ' No new line
print " 6 hello out there "
#ENDIF

close #1   ' Cons  - put/prints do not need flush
close #2   ' "CON" - This seems to flush "CON" prints
close #3   ' Scrn  - put/prints do not need flush
print " 7 HELLO out there "

a=2
s="23"
L = a + Val(s)

if L > 3 then
L = 3
Else
L = 1234
EndIf
Print
print "8 hello out there      888888 "+chr(13);  ' Cr returns to SOL, so NEXT line over-writes
print "9 HELLO OUT       999999"+chr(13);        ' Cr returns to SOL, so NEXT line over-writes
Print "10 press any key AA"
Sleep
' Test capture put ::  - #1, #3 immediate, but #2 has delay effect until close #2

'1 hello out there, Started JTest
'err=           0
' 2 Opened Cons as #1  4 put #3 statement 5 second half Cons  6 hello out there
' 3 put #2 statement 7 HELLO out there
'
'10 press any key AA9999988888

' Test Capture TestPrint - same as above.

'1 hello out there, Started JTest
'err=           0
' 2 Opened Cons as #1  4 print #3 statement 5 second half Cons  6 hello out there
' 3 print #2 statement 7 HELLO out there
'
'10 press any key AA9999988888



The above I was able to compile and debug (Step/watch) using this combination BAT file
"C:\FreeBASIC\FreeBASIC_1.05.0_Win32\fbc.exe" -g -v JTest.bas
"C:\FreeBASIC\fbdebugger292\fbdbg 32\fbdebugger.exe" C:\FreeBASIC\COM_tests\JTest.exe

but the 64b versions of those have some issues...
• Posts: 10,193
edited 2018-09-04 09:15
I've only been lightly perusing this thread out of interest but on the question of output devices I prefer to select a device for output and then whatever is sent goes to that output device. No need to have PRINT WITH etc. For instance in Tachyon I can simply redirect output from the serial console to say the VGA device and all output switches to that device. There is no need to hard code anything for a particular device and even system output is redirected too totally transparently. Same goes for input and files so all these work the same:
WORDS or VGA WORDS or LCD WORDS or 5 SERIAL WORDS to output serial on pin 5 or FILE> WORDS where output writes to the open file in the currently selected channel etc. The standard console is reset back with CON.

Dynamic strings are another thing, I'd be interested to see what you end up doing in this regard
• Posts: 8,126
ersmith,
I fully support this project. The more options the better.
I will be watching this thread to see how if works out.
Good luck,
Bean
• Posts: 14,475
Bean wrote: »
ersmith,
I fully support this project. The more options the better.
I will be watching this thread to see how if works out.
Good luck,
Bean
You guys are all going to be blown away once I roll out my new AdvSys2 language as a general-purpose Propeller programming language!
Don't worry yet though. At the moment it can't even blink an LED!
• Posts: 5,399
I've only been lightly perusing this thread out of interest but on the question of output devices I prefer to select a device for output and then whatever is sent goes to that output device. No need to have PRINT WITH etc.
WORDS or VGA WORDS or LCD WORDS or 5 SERIAL WORDS to output serial on pin 5 or FILE> WORDS where output writes to the open file in the currently selected channel etc. The standard console is reset back with CON.
FORTH has the advantage there of being a functional language, so composing words gives you a very powerful way to modify things. I guess the closest we'd get to that in BASIC would be to do away with PRINT and just implement a PRINT method on device objects, so something like:
   ser.print("hello, terminal")
vga.print("hello, screen")

Which is a fine way to do things, but not quite as aligned with "traditional" BASIC as I was hoping for.
Dynamic strings are another thing, I'd be interested to see what you end up doing in this regard

I think David has convinced me to go with reference counted dynamic strings. I had really hoped to avoid that, but the use case of:
function greet$(name$)
return "hello, " + name$end function  requires that functions be able to return temporary strings, which in turn means they can't always be allocated on the stack . On the other hand I think that's the only case where dynamic heap allocation is absolutely required. If we disallow it then implementing strings becomes vastly simpler. But again, the language wouldn't be "traditional" BASIC, nor as user friendly. Actually implementing the reference counting pretty much requires being able to run some code whenever a string is created, copied (decrementing the old reference, incrementing the new), or destroyed. The last one is the biggest rub, since it requires running destructors on objects going out of scope, e.g. temporary variables inside a function before the function returns. Which starts taking us into serious object oriented programming territory, but does open up some interesting other possibilities. The other alternative, a mark-and-sweep type garbage collector, might be simpler if the references are always held in HUB memory. But if we allow references in COG memory then it becomes impossible, because one COG can't read other COGs memory to see if there are references there. Maybe an extra layer of indirection, like a handle table, might avoid that problem. Lots to think about, anyway. Thanks everyone for your feedback. Eric • Posts: 5,399 Bean wrote: » ersmith, I fully support this project. The more options the better. I will be watching this thread to see how if works out. Good luck, Bean Thanks Bean! I definitely don't think of this compiler as a replacement for PropBasic, but rather as a parallel approach. I like what you did with PropBasic -- it's an elegant compiler, and great for people coming from the Basic Stamp to the Prop. I never did any Stamp programming though, so my mind is kind of stuck on old school 80's BASIC with a touch of Microsoft's later changes. Eric • Posts: 10,193 edited 2018-09-04 12:34 I have wondered about whether to bother or not with dynamic strings before but for me they are used a bit differently although having capable string functions probably really requires it. There are always all those equivalent functions to Basic for manipulating strings and one needs to allocate space, maintain that space as long as it's needed, and know when it's not. I've never given it much thought until now though. But I do know that a heap is not boundless so therefore whatever limit is determined basically means that as long as that limit had not been exceeded then leave it as it is. So when it comes time to make room my first thought is to have a fixed table of string pointers and attributes including last accessed time where my 32-bit ms runtime count would be used as the stamp and the oldest culled first. The new string could use that area that is released with the remainder moved up/down if necessary and the pointer table updated. Perhaps you guys are familiar with more sophisticated methods anyway and maybe I'm way off but that's just my thinking at the moment that I thought I'd share. (I will try this out and see how dumb or good it is I guess) However, I can't see why your output device couldn't just be revectorable and Basic only has to point the output vector to VGA and so all output is sent to that "device". I try to make all my devices handle streams rather than painfully calling methods, so rather than public vga.cr or vga.cls there is only vga.out which handles the stream of characters and controls etc which calls the internal private methods for vga.cr and so on (but normally vga.out is not called directly). The stream doesn't always have to be 8-bit although that is more convenient for printing. EMIT in Tachyon doesn't really do anything except determine what to pass the data to which by default is the serial console. Example: Imagine that you only have serial output but on the other end of that serial output are smarts that can redirect the stream to many different devices, including files. Your code just needs to send (or call) that magic word to switch the output device. • Posts: 8,126 edited 2018-09-04 12:54 ersmith wrote: » Bean wrote: » ersmith, I fully support this project. The more options the better. I will be watching this thread to see how if works out. Good luck, Bean Thanks Bean! I definitely don't think of this compiler as a replacement for PropBasic, but rather as a parallel approach. I like what you did with PropBasic -- it's an elegant compiler, and great for people coming from the Basic Stamp to the Prop. I never did any Stamp programming though, so my mind is kind of stuck on old school 80's BASIC with a touch of Microsoft's later changes. Eric PropBasic has it's roots in SX/B for the SX chip. If I was to re-write it today I would do things very differently. I would start with a good expression evaluation procedure. Yeah, I'm thinking I might get some good PropBasic enhancement ideas from your compiler too. I already like "PRINT WITH" I would probably use it something like: OutChar SUB 1 ' Character output Start: PRINT WITH OutChar, "Hello World!" END SUB OutChar SEROUT SOutPin, Baud, __PARAM1 ENDSUB  Bean • Posts: 5,399 edited 2018-09-04 13:45 So when it comes time to make room my first thought is to have a fixed table of string pointers and attributes including last accessed time where my 32-bit ms runtime count would be used as the stamp and the oldest culled first. The new string could use that area that is released with the remainder moved up/down if necessary and the pointer table updated. That seems a bit dangerous -- it would usually work, but there isn't any guarantee that the oldest string is never going to be accessed again, so one could certainly see scenarios where it would overwrite a string that gets used later. In the approach I mentioned above each string has a use count, which gets incremented when it is copied and decremented when the copy is no longer in use. When the count reaches 0 the string is freed. This approach works well for static data like strings; it can fail for data structures that can involve loops (because then you can get a loop in which each element points to another, so the use counts are nonzero, but the loop as a whole is no longer reachable). Another approach, mark-and-sweep, is a bit different. Basically when you run out of memory you walk through the heap and mark all objects as "free", then run through the stack and global variables and any time you see a reference to an object you re-mark it as "used". At the end you should have an accurate tally of used and free memory (anything no longer referenced will be marked "free") and you can then re-allocate from the new free space. However, I can't see why your output device couldn't just be revectorable and Basic only has to point the output vector to VGA and so all output is sent to that "device". I try to make all my devices handle streams rather than painfully calling methods, so rather than public vga.cr or vga.cls there is only vga.out which handles the stream of characters and controls etc which calls the internal private methods for vga.cr and so on (but normally vga.out is not called directly). Well, that's basically what PRINT WITH was intended to do -- it sets the vector used internally by PRINT. So basically PRINT WITH redirects the output to the new stream. I illustrated it with some methods, but any kind of function could be used as the argument for PRINT WITH. Internally PRINT would do all the formatting and then when it wants to output characters it would call the vector. I guess I didn't explain it very well. I guess we could rename PRINT WITH as OPEN and change around the syntax a bit to allow multiple vectors. Then we could do:  OPEN #0, vgafunc ' set default PRINT function OPEN #1, serfunc ' set alternate PRINT function #1 PRINT "hello, vga" PRINT #1, "hello serial" PRINT "hello again, vga" '' now revector using LCD instead of VGA OPEN #0, lcdfunc PRINT "now print is going to LCD instead of VGA" PRINT #1, "print #1 still goes to serial"  (so plain PRINT is an alias for PRINT #0) • Posts: 14,475 You may want to use handles for strings so you can compact string space. If a program does a lot of string manipulation the heap could get pretty fragmented. • Posts: 5,399 I'm still trying to figure out if it's possible to avoid heap allocation entirely. The main problem with stack based allocation is returning stack objects. But that problem goes away if we pass a pointer to the destination object, and use that for return values. That is, a function like: function greet$(name$) return "hello " + name$
end function

gets translated internally into:
sub greet_$(byref retval$, name$) retval$ = "hello " + name$end sub  Then in  a$ = greet$(u$)

we end up doing the C equivalent of:
   strncpy(a.ptr, "hello ", a.size);
strncat(a.ptr, u.ptr, a.size);

There will still be times when we'll have to allocate temporaries on the stack, e.g. in:
PRINT greet$(firstname$ + lastname$)  which will end up doing something like:  struct string temp1; temp1.size = firstname.size + lastname.size; temp1.ptr = alloca(temp1.size); strncpy(temp1.ptr, firstname.ptr, temp1.size); strncat(temp1.ptr, lastname.ptr, temp1.size); struct string temp2; temp2.size = min(temp1.size, DEFAULT_STRING_LEN); temp2.ptr = alloca(temp2.size); greet_(temp2, temp1); // result will be copied to temp2.ptr print(temp2);  • Posts: 14,475 ersmith wrote: » I'm still trying to figure out if it's possible to avoid heap allocation entirely. The main problem with stack based allocation is returning stack objects. But that problem goes away if we pass a pointer to the destination object, and use that for return values. That is, a function like: function greet$(name$) return "hello " + name$
end function

gets translated internally into:
sub greet_$(byref retval$, name$) retval$ = "hello " + name$end sub  Then in  a$ = greet$(u$)

we end up doing the C equivalent of:
   strncpy(a.ptr, "hello ", a.size);
strncat(a.ptr, u.ptr, a.size);

There will still be times when we'll have to allocate temporaries on the stack, e.g. in:
PRINT greet$(firstname$ + lastname\$)

which will end up doing something like:
   struct string temp1;
temp1.size = firstname.size + lastname.size;
temp1.ptr = alloca(temp1.size);
strncpy(temp1.ptr, firstname.ptr, temp1.size);
strncat(temp1.ptr, lastname.ptr, temp1.size);
struct string temp2;
temp2.size = min(temp1.size, DEFAULT_STRING_LEN);
temp2.ptr = alloca(temp2.size);
greet_(temp2, temp1); // result will be copied to temp2.ptr
print(temp2);

Clever. I tried to figure out how to do that with xbasic but didn't come up with that idea. Maybe I should go back and try it.

• Posts: 2,155
• Posts: 14,950
edited 2018-09-04 20:30
ersmith wrote: »
Well, that's basically what PRINT WITH was intended to do -- it sets the vector used internally by PRINT. So basically PRINT WITH redirects the output to the new stream. I illustrated it with some methods, but any kind of function could be used as the argument for PRINT WITH. Internally PRINT would do all the formatting and then when it wants to output characters it would call the vector. I guess I didn't explain it very well.

I guess we could rename PRINT WITH as OPEN and change around the syntax a bit to allow multiple vectors. Then we could do:
  OPEN #0, vgafunc  ' set default PRINT function
OPEN #1, serfunc   ' set alternate PRINT function #1
PRINT "hello, vga"
PRINT #1, "hello serial"
PRINT "hello again, vga"
'' now revector using LCD instead of VGA
OPEN #0, lcdfunc
PRINT "now print is going to LCD instead of VGA"
PRINT #1, "print #1 still goes to serial"

(so plain PRINT is an alias for PRINT #0)

I like the approach of keeping this broadly compile compatible with FreeBASIC, as that means users can instantly use any existing FreeBASIC IDE / Compilers / Debuggers as development

eg Here I tested named ports, and this code compiles and runs in FreeBASIC.
Needs the prefix #, in Print, (not mandatory in Open..)
but does allow PRINT #vgafunc, Print,list which is easier to maintain and follow, but keeps compatible with FreeBASIC.
' Blank line added Derived from JTest, this adds Const names for ports
Dim a as Integer
Dim ern as Integer
Dim s as String
Dim L as Long
Const AS Long cCons = 1, cCON = 2, cScrn = 3

print "1 hello out there, Started JTestC"
' Example FB COM port syntax
'If Open Com ("COM16:115200,n,8,1,cs0,rs,ds0,cd0,bin,op2000,TB32000,RB32000" For Binary As #1) <> 0 Then  'generic baud
The common typo/error case of duplicate open (same port#) may be able to be trapped/reported at compile time ?  in FreeBasic it compiles ok, but gives err=1 on run.

' In FreeBASIC Cons,"CON" and Scrn are reserved keywords, and all can allow redirect #FileNum to screen
print "cCons",cCons
open Cons for output as cCons  ' Works  # here looks optional

print "cCON",cCON
open "CON" for output as cCON  ' Works but delays until close

print "cScrn",cScrn
open Scrn for output as cScrn    ' Works

ern=err
print "err=",ern    ' invalid cScrn = 2 gives err=1, valid 3,5 etc gives err 0
#define TestPrint
#IFDEF TestPrint
print #cCons," 2 Opened Cons as ",cCons;   '# here is needed, to separate port# from a print param.
print #cCON," 3 print as",cCON;        ' No new line
print #cScrn," 4 print as",cScrn;      ' No new line
sleep 2000
print #cCons," 5 second half Cons ";        ' No new line
print " 6 hello out there "
#ELSE  'Test put
put #cCons, ," 2 Opened cCons as ",cCons
put #cCON, ," 3 put cCON statement"         ' No new line
put #cScrn, ," 4 put cScrn statement"         ' No new line
sleep 2000
put #cCons, ," 5 second half Cons "         ' No new line
print " 6 hello out there "
#ENDIF

close #cCons   ' Cons  - put/prints do not need flush
close #cCON    ' "CON" - This seems to flush "CON" prints
close #cScrn   ' Scrn  - put/prints do not need flush
print " 7 HELLO out there "

a=2
s="23"
L = a + Val(s)

if L > 3 then
L = 3
Else
L = 1234
EndIf
Print
print "8 hello out there      888888 "+chr(13);  ' Cr returns to SOL, so NEXT line over-writes
print "9 HELLO OUT       999999"+chr(13);        ' Cr returns to SOL, so NEXT line over-writes
Print "10 press any key AA"
Sleep    'Wait for key

' Test captures ::  - #1, #3 immediate, but "CON" has delay effect until close #2

' Test Capture TestPrint - same delay effect on "CON" as above.

'1 hello out there, Started JTestC
'cCons          1
'cCON           2
'cScrn          3
'err=           0
' 2 Opened Cons as            1 4 print as    3 5 second half Cons  6 hello out there
' 3 print as    2 7 HELLO out there
'
'10 press any key AA9999988888


This variant of the same code uses FreeFile(), and it also compiles and runs - I'm less sure if FreeFile() is needed in a Prop Basic ?
REM Blank line added Derived from JTest, this adds names for ports
Dim a as Integer
Dim ern as Integer
Dim s as String
Dim L as Long
Dim fCons As Long,fCON As Long, fScrn as Long

print "1 hello out there, Started JTestN"
' Example FB COM port syntax
'If Open Com ("COM16:115200,n,8,1,cs0,rs,ds0,cd0,bin,op2000,TB32000,RB32000" For Binary As #1) <> 0 Then  'generic baud

fCons = FreeFile
print "fCons",fCons
open Cons for output as fCons  ' Works

fCON = FreeFile
print "fCON",fCON
open "CON" for output as fCON  ' Works but delays until close

fScrn = FreeFile
print "fScrn",fScrn
open Scrn for output as fScrn    ' Works

ern=err
print "err=",ern
#define TestPrint
#IFDEF TestPrint
print #fCons," 2 Opened Cons as ",fCons;
print #fCON," 3 print #2 statement as",fCON;        ' No new line
print #fScrn," 4 print #3 statement as",fScrn;      ' No new line
sleep 2000
print #fCons," 5 second half Cons ";        ' No new line
print " 6 hello out there "
#ELSE  'Test put
put #fCons, ," 2 Opened Cons as #1 "
put #fCON, ," 3 put #2 statement"         ' No new line
put #fScrn, ," 4 put #3 statement"         ' No new line
sleep 2000
put #fCons, ," 5 second half Cons "         ' No new line
print " 6 hello out there "
#ENDIF

close #fCons   ' Cons  - put/prints do not need flush
close #fCON    ' "CON" - This seems to flush "CON" prints
close #fScrn   ' Scrn  - put/prints do not need flush
print " 7 HELLO out there "

a=2
s="23"
L = a + Val(s)

if L > 3 then
L = 3
Else
L = 1234
EndIf
Print
print "8 hello out there      888888 "+chr(13);  ' Cr returns to SOL, so NEXT line over-writes
print "9 HELLO OUT       999999"+chr(13);        ' Cr returns to SOL, so NEXT line over-writes
Print "10 press any key AA"
Sleep    'Wait for key

' Test captures ::  - #1, #3 immediate, but #2 has delay effect until close #2

' Test Capture TestPrint - same as above.

'1 hello out there, Started JTest
'fCons          1
'fCON           2
'fScrn          3
'err=           0
' 2 Opened Cons as            1 4 print #3 statement as     3 5 second half Cons  6 hello out there
' 3 print #2 statement as     2 7 HELLO out there
'
'10 press any key AA9999988888


• Posts: 14,950
Here is an example of how compatible #IFDEF and compile, could support dual platform development and simulation/debug, via FreeBASIC IDE/Debuggers
#IFDEF BlockComment
Comment section
we can
type
anything inside BlockComment build from # IFDEF

#ifdef __FB_VERSION__
If Open Com ("COM16:115200,n,8,1,cs0,rs,ds0,cd0,bin,op2000,TB32000,RB32000" For Binary As cSerialA) <> 0 Then  'generic baud
n = Err()
Print "unable to open serial port, (press any key) Error Code: ";n
Sleep
End
else
n = Err()
Print "No error on Open Com, Error Code: "; n  'always 0
End If
open Cons for output as cCons  ' screen reporting, or can be another serial open...

#else  'This is PropHW path - whatever Serial open is used in Prop...
Open PropCom (31,32,115200,n,8,1) For Binary As cSerialA
Open PropCom (1,2,115200,n,8,1) For Binary As cCons
#endif

Print #cSerialA,"SSS"  ' works in either platform
Print #cCons,"CCC"     ' works in either platform

#ENDIF

• Posts: 18,062
Excellent news. I've always wanted to see a more general Basic on P1. Having a compatible P1 and P2 Basic would be nice.