Shop OBEX P1 Docs P2 Docs Learn Events
"Small C" Compiler - Page 2 — Parallax Forums

"Small C" Compiler

2

Comments

  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-27 16:13
    Now that I have a spin stamp, I wrote a M4 file for the smallc compiler.
    First I recompiled the smallc compiler to accept names of 32 characters length
    (default is 8).
    I made two test programs and also 2 batch files, so you just edit a C test file,
    then doubleclick the batchfile and the spin source is created.

    The M4 file PROTOKEN.M4 converts the M4 output from the smallc compiler
    into a spin file that consists of a single method start that interprets the bytecodes.
    Note that the actual code that deals with the bytecodes is not in place.

    Just unzip the zipfile in some directory.
    Look at proptst1.spin (which is also in the zip) to see how the spin source comes·out.

    Since rcsc has builtin keywords for task and interrupt, I think I can use cognew to start a task.
    For interrupt I think an interval timer could do the job.

    regards peter
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-27 17:14
    Peter , that looks most interesting.. Can you elaborate a little bit how this system works in the first place, what you have exactly done to adapt ist, what you plan to do with it in the next monthes,...??

    Perhaps make a comprehensive wiki page of it smile.gif
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-27 17:44
    The Retargetable Concurrent Smallc Compiler (rcsc) is basically used as is,
    although I recompiled it to support variable/function names of 32 characters.
    This compiler outputs pcodes that you can inspect in proptst1.m4 for example.
    The macro assembler M4 then converts these pcodes into whatever you want.
    Protoken.m4 converts the pcodes into byte sequences that·are to be interpreted
    by the start method. It is also·possible to write an M4 file that converts the pcodes
    directly into spin statements but·the BIG problem there·is the indenting of lines.
    So I settle for an interpreter.
    Another nice feature of rcsc is the builtin support for concurrent tasks and interrupts.
    A task is just a C function that executes concurrently with other code. A cog is
    just ideal to run a task.
    Missing are I/O routines but I have some of these that I wrote for the SX chip.
    I plan to have at least pin control and debug output. That probaly will take the
    form of a BIOS call (as seen from C), which should allow me to pass some
    register values as well. Once I have that all in place, there is a working base
    for the rcsc compiler.

    regards peter

    ·
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-27 18:39
    Hurra!!!

    This looks *very* promising!
    I just tried a bit.
    Also tried to include some assembler (got it to compile and to Spin-ify it), but had no success in seeing results.

    Have you got a simple example for #asm?

    Do you want to tell us, that there is a cognew-command? smile.gif
    Even COG-ing C-functions?

    And where do I find a reference of the C-dialect?


    THANKS for your work!
    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-27 19:29
    Nick,
    On page 1·of this thread, the 6th message, is an·attachement with pdf documents
    regarding the Concurrent and Retargetable Concurrent smallc. (Read them in that order).
    There is builtin support for tasks (and semaphores).
    A task is simply defined as

    task somefunction() {
    }

    The only libraries currently present are in the latest zip and are mostly text handling functions.
    Anything else has to be written (or imported from various sources).
    I meant to say that a task function should execute in a seperate cog·but that is not readily available.

    I will first write code that makes the start method do something. As it is all spin I can
    use a serial driver for debug output (that would be the standard output I think).

    regards peter
  • hippyhippy Posts: 1,981
    edited 2007-12-27 23:47
    Peter Verkaik said...
    The macro assembler M4 then converts these pcodes into whatever you want.
    Protoken.m4 converts the pcodes into byte sequences that are to be interpreted
    by the start method. It is also possible to write an M4 file that converts the pcodes
    directly into spin statements but the BIG problem there is the indenting of lines.
    So I settle for an interpreter.

    Lots of exciting things going on here by the looks of it. One option for the conversion would be to Spin Bytecode which gets round the problem of indentation and ( for trying to convert any non block structured language to Spin ) the lack of a GOTO instruction. That also saves having to write an interpreter.

    I'd say get the compiler generating the right pcodes, then worry about translation / interpreter second.
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-28 00:06
    > I meant to say that a task function should execute in a seperate cog but that is not readily available.

    Yes. I realized that now.
    Preemptive means some multi-tasking scheme. So there should be a task-switcher that sequences through the different tasks.
    If you could give me a link to a better description than the two PDFs, I'd like to read more. Especially about the P-machine and how many instructions there are.
    *IF* a P-machine would fit into one COG, and the preemptiveness replaced, then that compiler could generate perfect code for the Propeller. Maybe there's also a non-preemptive version and this one would be a better start.


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-28 00:47
    I played a little bit with the compiler.... First findings:

    - None of my C-programs would compile unchanged. Most changes are extremely annoying and time consuming. This however will not impact any new programs. Those might even compile on modern ANSI-C compilers...

    - The generated intermediate code is quite standard. There is in fact no need for M4, but a hand made code generator can make optimized machine code; keeping the stack directly addressed in the COG. This might prove useful for small drivers, whithout hard timig constrains.

    - I still have no idea how the linker looks like...

    - High level optimization is quite limited (no common expression identification, no loop invariants optimization,....); but it precomputes constant expresions smile.gif
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-28 13:45
    First update.
    I've ·put the rcsc interpreter into its own rcsc.spin file.
    That means the protoken.m4 needs no editing except for adding pseudo pcodes
    (not generated by rcsc) and some bugfixing if necessary.

    @Nick,
    You can't just put PASM statements between #asm and #endasm as the interpreter
    doesn't recognize them.
    Anything placed between #asm and #endasm is directly written to the output file of rcsc.
    (I used it to output DAT prior to declaring system ports RADIR0-RADIR3 and RA0-RA3 in prop.h)
    What you can place between #asm and #endasm are pcodes. These could be the·rcsc generated
    pcodes but more useful are pseudo pcodes that we define ourselves.
    This provides the glue logic to call native code.

    somefunction(par) int par; {
    #asm
    MyNativeFunction(par)
    #endasm
    }

    All that is needed is to put another macro (with name MyNativeFunction) in protoken.m4
    and add the pMyNativeFunction pcode to the interpreter. The interpreter can then
    call the code that performs the function (cognew for example to start a task).

    @deSilva,
    Using M4 gives quicker results than modifying the rcsc compiler. Modifying the compiler
    might even lead to incorrect output (the rcsc compiler is known to work).
    The linker LINK.EXE is included in rcsc package (as is the assembler ASM.EXE).
    (for the package see the 6th message on the first page of this thread).


    I have added the interpreter code for most pcodes (see rcsc.spin). The only pcodes not done are
    those that relate to the task and interrupt keywords of rcsc.
    Next I will add some basic functions using the method I described above.


    regards peter
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-28 14:09
    > You can't just put PASM statements between #asm and #endasm as the interpreter
    > doesn't recognize them.

    OK, a different level of ASM. smile.gif
    But it compiled nice. No matter that it would require to start a COG with it and pass the PAR. Doesn't sound like a too complicated task.

    No, I'm not pushing you!


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-28 14:15
    It is not required to start a new cog, I just mentioned it to point
    out that you call any native function (that you program in spin or pasm)
    by adding self defined pcodes and all that without modifying the compiler.

    regards peter
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-28 14:22
    You'll have problems trying to let run the Prop-ASM in HUB-RAM. wink.gif


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-28 14:28
    Peter Verkaik said...
    Using M4 gives quicker results than modifying the rcsc compiler. Modifying the compiler might even lead to incorrect output (the rcsc compiler is known to work).
    The idea was to take a separete program reading th eintermediate code (what M4 does). As th equalkity of the intermediate code is "average" it needs at least "peep-hole optimization" where macro generators are not good with smile.gif You also can also generate some important statistics and tables...

    Using a macrogenerator is generally a "quick-n-dirty" feasibility study, with the chance to abandon the project without too high losses...
    said...
    The linker LINK.EXE is included in rcsc package (as is the assembler ASM.EXE).
    (for the package see the 6th message on the first page of this thread).
    Sure, for ASM smile.gif That was not my question. It should also be possible to "link" the intermediate code files. As Nick put it: "another level of assembly code".

    If such linker does not exist this is something a code generator can also do. The work around is of course to include ALL C-code. The main disadvantage being that you will generally also include things not really needed.

    BTW: Have you had a look at this SmallC derivate? It contains a much more complete impletation of C...
    www.jennaron.com.au/smallc/smallc.html

    Post Edited (deSilva) : 12/28/2007 2:36:36 PM GMT
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-28 14:31
    Nick Mueller said...
    You'll have problems trying to let run the Prop-ASM in HUB-RAM.
    Not really.. That is what LMM does in some way. You just have to notify another COG to "pick up" this machine code. The tricky work is to define and check the needed interfaces (parameter passing, COG address allocation,...)
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-28 15:01
    @deSilva,
    Yes, I did look at that version, and at others (there even is a 32bit version for intel386 chips).
    The avr version is really targeted towards avr now, way too complicated to change it.
    Besides, I like it that the rcsc outputs pcodes, makes it much easier to convert using M4.
    The rcsc has a builtin peephole optimizer (that could be improved though).

    You can put functions in libraries. Then only the functions referenced are included
    in the compilation which makes it unnecessary to have a linker.

    @Nick, there is no PASM code running, just spin for now. Native functions can also
    be just spin code.

    regards peter
    ·
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-28 15:35
    > You just have to notify another COG to "pick up" this machine code.

    Sure. That's what I said.

    > Nick, there is no PASM code running, just spin for now.
    > Native functions can also be just spin code.

    You misunderstood me. To let run *Prop*ASM, it has to run in a separate COG.

    To reduce confusions, we should use:
    PropASM for Propeller ASM code
    and VirtualASM (or VASM, or PCodeASM; pick one) for assembler with p-code (-> NOT PASM)


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-28 16:30
    It is a sidetrack, I know.
    But so many pictures from days long gone popped up. I had to search the web for UCSD-Pascal and the p-Machine.
    Decades ago, I knew Apple ][noparse][[/noparse]'s p-machine inside out (and also made some changes to speed it up (2%!!! smile.gif).

    Some links with interesting readings about how such a beast works:
    <http://www.threedee.com/jcm/psystem/&gt;
    <http://ucsd-psystem-xc.sourceforge.net/&gt;
    <http://miller.emu.id.au/pmiller/ucsd-psystem-um/&gt;
    <http://www.hansotten.com/indexpascal.html&gt;
    <http://en.wikipedia.org/wiki/UCSD_p-System&gt;


    Unfortunatly, I no longer have the most interesting book that was an analysis of the UCSD p-Machine on the Apple.

    Maybe I still have that poster: <http://pascal-central.com/pascal-syntax.html&gt;
    But I had to stick it back together, because my girlfriend of that time tore it apart.
    Seems I have spent to many hours in front of the computer. smile.gif

    Sorry for the diversion

    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-28 18:04
    In this century there is also a lot of reading about Java VM and JIT compilation smile.gif
  • CardboardGuruCardboardGuru Posts: 443
    edited 2007-12-28 18:23
    deSilva said...
    In this century there is also a lot of reading about Java VM and JIT compilation smile.gif

    Ah! But stuff from last century is often more applicable when programming in 32K memory. Not much room there for JIT or JVMs. wink.gif

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Help to build the Propeller wiki - propeller.wikispaces.com
    Play Defender - Propeller version of the classic game
    Prop Room Robotics - my web store for Roomba spare parts in the UK
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-28 22:37
    I have some trouble retrieving runtime addresses.
    The generated table (starting at _main) holds label references but these
    are compiletime offsets.

    How can I calculate the runtime addresses?
    The addresses in the table are stored using @
    The main program retrieves the runtime address of _main (via method getMain)
    and passes this value to the rcsc object.

    regards peter
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-29 12:17
    Problem resolved. I solved it by storing address differences (eg. instead of storing the
    address of a function, I store @function-@_main) and adding the runtime address
    for _main to the difference to find the runtime address of the function.

    This version includes proptst3.c that has a printf function. You need
    the MultiCogSerialDebug object (which serves as standard output).
    Now that we can print anything we can really start testing.

    The program rcsc_test.spin is the top object file.
    The picture is the proptst3 output.

    regards peter
    655 x 358 - 38K
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-29 12:36
    > In this century there is also a lot of reading about Java VM and JIT compilation smile.gif

    I'm too old for that! wink.gif
    I don't know exactly, but I think that the basis for a virtual machine was layed by N. Wirth in his excellent booklet "Compiler Construction". It's really worth reading (it's not so much to read) and describes the whole process in an incredibly readable and simple way. Someone posted a link to the free version as a PDF (the book is quite old).
    The USCD-Pascal is based (I guess) on that idea and does exactly what Spin does with its VM.

    VMs arent fast (see Spin), but they produce quite dense code, because you can tailor the virtual CPU to the task.
    The interesting thing behind this Small-C project (*clap*clap*clap*) is that the VM is emulated by another VM (Spin). That really makes my smile! Not meaning that I laugh about it!

    It would be interesting to see wether the VM can be implemented in PropASM ...


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • hippyhippy Posts: 1,981
    edited 2007-12-29 13:23
    Nick Mueller said...
    It would be interesting to see wether the VM can be implemented in PropASM ...

    Certainly looks possible and fairly easy to do, and from the looks of it should very fast to execute each P-Code.

    The way I'd approach it is to take the SpinVM work I'm currently doing and add RCSC handling to it. Should be easy enough to have SpinVM switch from interpreting Spin bytecode to RCSC Bytecode. The big advantage to that is it would be possible to call Spin methods from RCSC which would give access to Spin objects, and vice-versa, Spin could call C routines, although more tortuous because it means having to use what the Spin language gives to do that.

    Ultimately the RCSC compiler should have a means to support linking with a Spin program but it can be hacked together with what we currently have available.

    The rcsc.spin interpreter seems to provide all the necessary information, but is there a definitive description of the RCSC P-Code anywhere ?

    PS : Excellent work Peter.
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-29 13:51
    Converting rcsc.spin into propasm is the ultimate goal, but my knowledge of propasm
    is yet too small to do that in the most effective way.·And best to postpone that
    until rcsc.spin is fully working on all pcodes.
    I will first write some real test program that tests·all the logical and math operators to find out
    which pcode parts in rcsc.spin needs adjusting (many still use the compiler offsets
    rather than the runtime addresses).
    The description of the pcodes as stated in rcsc.spin is all that is required as it
    explains exactly what a pcode is supposed to do. All available pcodes are in there.
    Any new pcodes are the ones I made up (pNOP,pCCARGC,pPRINTF) and all these
    are intended to call a native function. One thing I want to do is that the protoken.m4
    recognizes the use of RA0-RA3 and RADIR0-RADIR3 and then output some pcode
    that updates the real RA and RADIR so we have direct access to the hardware pins from
    smallc.
    The current setup is such that a rcsc source compiles into a spin object rather than
    a top object. That provides the most flexible use I think, as you simply use the
    object as I did in rcsc_test.spin

    regards peter
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-29 14:13
    > Converting rcsc.spin into propasm is the ultimate goal, but my knowledge of propasm
    > is yet too small to do that in the most effective way. And best to postpone that
    > until rcsc.spin is fully working on all pcodes.

    That's absolutely the right track you are on!
    There will be others that fill in the PropASM-part for you. I guess.

    **Great** work you are doing at a huge pace!

    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-30 13:30
    New update. As I expected many pcode parts needed tuning but I think
    I got them all right now. The output shows all expected values for all the
    statements in the proptst4.c test file.

    Letting M4 detect predefined names for RA and RADIR is not used as RA and RADIR
    are not mapped into the ram addresses (in the SX the ports are mapped to ram addresses
    so there I used predefined variables fixed at the port addresses).
    So I defined a number of routines to write and read bits, bytes and words to RA and RADIR.
    At the moment the new pcodes are·not converted by M4 and so these show up as labels
    in·the generated spin code (proptst4.spin).

    I made up some logical names for these routines but I rather use some widespread names,
    like used in ANSI C or PC99 (or is that C99?)

    If anyone can direct me to a good source/manual for these routines (I expect names like
    readport, writepin etc.) that would be a great help.

    regards peter
  • Nick MuellerNick Mueller Posts: 815
    edited 2007-12-30 17:07
    > like used in ANSI C or PC99 (or is that C99?)

    That would be C99. But a tiny C can't be a C99.

    > If anyone can direct me to a good source/manual for these routines (I expect names like
    > readport, writepin etc.) that would be a great help.

    A C(99) book?

    You won't find functions like readport etc. there. Not part of the language, not in the standard libraries. But I'd download a manual from Imagecraft, Codewarrior or such for some inspiration. They do C for µCs.


    Nick

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Never use force, just go for a bigger hammer!

    The DIY Digital-Readout for mills, lathes etc.:
    YADRO
  • hippyhippy Posts: 1,981
    edited 2007-12-30 17:54
    Just to throw some extra ( and unnecessary at this time ) work onto your plate; the stack in rcsc.spin runs down from top of memory ( $FFFF towards $0000 ), is byte-stepping and leaves SP pointing to TOS-1. Ideally for optimised PASM it would be better if the stack was long stepping and for integration with Spin grew towards top of memory.

    Your own -

    byte[noparse][[/noparse] ax+0 ] := bx.byte[noparse][[/noparse] 0 ]
    byte[noparse][[/noparse] ax+1 ] := bx.byte[noparse][[/noparse] 1 ]

    style sequences would probably also benefit from becoming "word[noparse][[/noparse]ax] := bx" although there may be endian issues I've not considered.

    I created an RCSC interpreter skeleton which ran with my own SpinVM which would allow Spin to invoke RCSC - CogInit(-1,@Pcode). The problem I have is in not knowing anything at all about RCSC code generation so it's not clear to me how the runtime stack is interfaced with. BX and CX are used as index / offsets but I'm not clear what they are indexed / offset from and so on.

    Without a detailed description of the RCSC VM I don't think it's going to be easy for someone not already well familiar with it to independently implement an optimised PASM version beyond simply translating your own Spin code line-by-line.

    On the plus side, the experiment shows that implementing any arbitrary P-Code interpreter isn't that hard with the right framework to start from and that if lack of VM description or things like 32-bit variables prove a problem to add, any other P-Code generating language could be used, and we're not just limited to C; Pascal, possibly even Java of some sort.

    That's not in anyway a criticism of what you're doing which is extremely worthy in itself. My advice would be -

    * Keep heading in the direction you are doing
    * Add 32-bit variable support
    * Optimise the rcsc.spin
    * Convert to PASM.

    A faster PASM version is just icing on the cake.
  • hippyhippy Posts: 1,981
    edited 2007-12-30 18:01
    One other thing; rather than having the P-Code opcodes numbered 0 to N where how many operands there are is determined by position in that list, opcodes of the form %ooiiiiii would be something I'd consider, where 'oo' indicates how many operands -

    00iiiiii - None
    01iiiiii - One
    10iiiiii - Two
    110iiiii - Three
    111iiiii - Four

    That would make it easier to add new opcodes at the end of existing groups without having to make a major change of renumbering the whole list.
  • Peter VerkaikPeter Verkaik Posts: 3,956
    edited 2007-12-30 19:46
    Hippy,

    Can't change the way the stack must behave without rewriting the rcsc compiler.
    The rcsc compiler generates ADDSP pcodes at the start of a function and after a function returns
    (to allocate and free space used for locals). Then there are also offsets that are used to access
    locals, and these not only occur when accessing locals directly. When passing a pointer to a local
    (eg. its address) then this address is calculated with respect to the stack pointer frame,
    which is not·word aligned.

    Regarding the pcode values, I considered something like your scheme but since the most pcodes
    take 0 or 2 operands I rejected that. The current values have gaps which allow easy adding
    of pcodes with any number of operands (new added pcodes always take 0 operands as parameters
    are passed via the stack) but I may change that as well. If possible I want the protoken.m4
    to be changed only when the compiler changes and not when adding a native function.

    One way of doing that is to define a single new pcode pNATIVE that takes 1 operand, the
    index in a native list of routines. For example, the printf() then would become
    printf(format,value) char *format, int value; {
    #asm
    pNATIVE(1)
    #endasm
    }

    I tried to support longs by redefining two rcsc compiler constants: BPW (bytes per word)
    and LBPW (log2(BPW)). Integers would then be 4 bytes so basically longs.·It compiled but the
    recompiled version did not run properly so there must be something in the rcsc source that
    somehow depends on BPW being 2. For now, the support is therefore limited to bytes and words.


    regards peter


    ·
Sign In or Register to comment.