Propeller ROM source code HERE

cgracey · 2008-02-24 23:17

Here·are all the source codes for the executable portions of the Propeller's ROM.

The booter and interpreter·binaries·appear scrambled when the ROM is read normally, but automatically unscramble when loading into a cog. Since Hippy succeeded in reversing the scrambling pattern in response to my·challenge, I'm revealing the source codes. I·hope this will yield more progress than headaches.

Note that the executable portion of the ROM ($F004-$FFFF)·was pre-filled with longs randomly picked from the interpreter's scrambled binary before the actual code was laid in. This was a feeble measure to obscure the boundaries of the interpreter and booter codes. It was not much of a deterrent to Hippy, however, as I'm not even sure that he noticed.

Here is a map of the executable portion of the ROM:

$F004 - Interpreter binary (scrambled)
$F800 - Booter binary (scrambled)
$FF00 - Copyright message (ascii, 32 columns/row)
$FF70 - Runner bytecode (Spin code which builds initial stack frames for Spin processes)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

Chip Gracey
Parallax, Inc.

dfletch · 2008-02-24 23:44

This is amazing! Thank you Hippy!

Are we allowed to cherry pick stuff like the math operations for use in our own projects?

Maybe someone will start an open source spin compiler project now. If so, count me in!

Cheers,

--fletch

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

cgracey · 2008-02-24 23:51

dfletch said...

Are we allowed to cherry pick stuff like the math operations for use in our own projects?

Of course you can! It's there to learn by and get utility out of.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

Chip Gracey
Parallax, Inc.

potatohead · 2008-02-24 23:54

Chip, you are just great.

If somebody makes a headache over this, bad on them.

Appreciated and have a great weekend.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Propeller Wiki: Share the coolness!

Chat in real time with other Propellerheads on IRC #propeller @ freenode.net

Post Edited (potatohead) : 2/25/2008 4:17:16 AM GMT

dartof · 2008-02-25 07:11

BRAVO! This was the missing piece that make this processor fully understandable like the CPUs of the 70's and 80's. I had put a small effort into unscrambling the code last year by programming a small disassembler with a bit unscrambler, but gave up due to shear number of permutations. Kudos to Hippy and others for the brilliant effort to get the right bit order and to Chip for sanctioning the effort. This has made my day.

Rayman · 2008-02-25 14:24

Nice.· Maybe·somebody will write us a Spin based Spin compiler now?

stevenmess2004 · 2008-02-25 20:10

Rayman said...
Nice. Maybe somebody will write us a Spin based Spin compiler now?

If nobody else starts than I might start one in the next couple of months. I want to finish a few other things first though... If I do I will be asking for help from everyone as it will be the first compiler that I have done.

hippy · 2008-02-25 20:32

To be honest, a command line compiler suitable for porting to non-WinXP platforms would probably be more appreciated and useful than one written in Spin.

Writing a Spin compiler never required the ROM Interpreter code to be known but if that's the spur for people to embark on such a project having it is a good enough excuse.

Phil Pilgrim (PhiPi) · 2008-02-25 21:16

Spin may be a difficult language to write a compiler in. It's not impossible, but Spin's minimal string-handling capabilities will certainly make things awkward. However, if this can be done, a nice add-on would be a multi-threaded Spin byte-code interpreter that can run on a PC. For maximum portability, it should be written in Perl, Python, or some other thread-safe, multi-platform language (i.e. not VB6). It would be slower than Propeller-based Spin, certainly, but at least the Spin-compiler-in-Spin could run on other platforms besides the Propeller.

-Phil

VIRAND · 2008-02-25 21:55

A Spin editor and compiler (plus PASM) running on the Propeller w/o PC would be awesome.
As examples, I love the PC-independent nature of femtobasic...And also 8052AH-BASIC.

That reminds me, it seems to me that ATARI 800 BASIC handled strings similarly to Spin,
and I once wrote a compiler in it.

Post Edited (VIRAND) : 2/25/2008 10:15:15 PM GMT

Jasper_M · 2008-02-25 22:11

I've updated some info to the Propeller Wiki based on the source code. propeller.wikispaces.com/Spin+Byte+Code see if you can find any inaccuracies / something that's just plain wrong. And please do edit it, as it's not the simplest explanation of opcode structure probably. I'll keep updating the wiki on the subject as I learn.

hippy · 2008-02-25 23:18

Phil Pilgrim (PhiPi) said...
For maximum portability, it should be written in Perl, Python, or some other thread-safe, multi-platform language (i.e. not VB6).

But ... but ... but ... that's my favourite programming language of choice ! You are however entirely right.

Bob Lawrence (VE1RLL) · 2008-02-25 23:36

Somebody said...
But ... but ... but ... that's my favourite programming language of choice !

Maybe you can use your VB with Mono

The Mono Project on Feb. 20 announced that it has developed a Visual Basic compiler that will enable software developers who use Microsoft Visual Basic to run their applications on any platform that supports Mono, such as Linux, without any code modifications.

Website:
www.linuxdevices.com/news/NS9725385854.html

Mono Project website:
www.mono-project.com/VisualBasic.NET_support

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Aka: CosmicBob

dfletch · 2008-02-25 23:42

I'm thinking of starting a C project using Bison for the parser. Lexers and parsers written in procedural code or OO always seemed very clumsy to me after learning Flex/Bison

Just not 100% sure at the moment about Flex and unicode. May have to write a custom lexer instead of Flex.

I'll keep you posted on my progress. Come to the IRC channel if you want to chat.

Cheers,

--fletch

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

Phil Pilgrim (PhiPi) · 2008-02-26 00:58

Hippy said...
But ... but ... but ... that's my favourite programming language of choice !

I hear ya. I thought VB3 was the cat's meow. Then I "upgraded" to VB6 and thought, "OMG! They ruined it! It's GINORMOUS!" That's when I switched to Perl and haven't looked back. It's an acquired taste, though, and the shortest programs can produce huge executables when run through perl2exe. One reason is that, due to the eval function, the embedded interpreter has to include the compiler, too. But Perl's expressiveness (if you can read it a month down the road) and text-handling capabilities are legendary. Plus, it runs on almost any platform.

Hippy, have I whetted your appetite any?

-Phil

Phil Pilgrim (PhiPi) · 2008-02-26 01:12

dFletch,

I wouldn't worry too much about the Unicode thing. It has always seemed a rather steep price to pay just to get a few graphics widgets onscreen anyway. I'm sure most people don't care and would be fine with ASCII. When I was doing my preprocessor in Perl, all the funny characters got replaced with question marks, and I thought, "Hey, that works!"

-Phil

JT Cook · 2008-02-26 01:55

Chip:

Looking at the source for the booter, it had the title for the micro as PNut. I remember early during the Hydra dev that Parallax was trying to deside on a name for the chip, but I was wondering how it got the name PNut?

dfletch · 2008-02-26 02:20

Good point, Phil. I'll leave unicode support for "Phase 2". Isn't that what PHB's usually say when a feature gets squeezed out due to other priorities?

I'm thinking I'll put the parser alone in it's own shared library. It would build and return a tree and have some nice high level functions e.g. spinParseFile(...). The compiler part can then simply use the parsing library and focus on output. Other projects could then use this lib without the compiled output for other purposes (e.g. syntax highlight, auto-complete, etc.)

Cheers,

--fletch

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

hippy · 2008-02-26 02:23

@ Phil : I guess I have to now own up to still using VB3. What's amazing is that the same code which compiles under VB3 in the blink of an eye takes seconds to minutes with VB6.

I have a quite bizarre development cycle. I usually code in PowerBasic for MS-DOS as 'the lowest common denominator', and I have a home-grown program which auto-ports to VB3 ( for 16-bit ) and VB6 ( for 32-bit ). The GUI front-ends are written in native VB3 ( and again auto-ported to VB6 ) or written in native VB6 for the specific application. My program also auto-ports to Perl for platform independence but I seem to have a bug in that at present which is proving to be a pain to track down. It was working.

Theoretically it can also port to REALbasic, C# and VB.Net but I've not maintained the libraries to do that recently.

It means adopting a particular coding style but it does have the advantage that the desirability of splitting the GUI front-end from the back-end and platform specific hardware interfacing is absolutely enforced. It also means minimal maintenance when the back-end changes.

@ JT Cook : "Peanut" ? Or maybe "nut" is a pun on kernel; "Propeller Kernel" ?

cgracey · 2008-02-26 02:59

JT Cook said...

Chip:

Looking at the source for the booter, it had the title for the micro as PNut. I remember early during the Hydra dev that Parallax was trying to deside on a name for the chip, but I was wondering how it got the name PNut?

PNut was what I called the Propeller project during development, for lack of·a better name. None of us here knew what to name the chip, until·the very end. This was a frustration to me, because I usually have well-formed ideas about such basic things. We held an internal naming contest, and I think it was my brother Ken that came up with "Propeller". Then, in drawing various logos containing propellers, the hat idea popped up, which made everything else make sense.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

Chip Gracey
Parallax, Inc.

dfletch · 2008-02-26 07:52

I've gotten an interesting start on an open source Spin compiler. I wrote a (hopefully complete!) LALR grammar for Spin language.

This is a rough first pass, but if anyone's interested I'm attaching what I've got so far. I'd love to hear opinions on the structure and I'm sure I've missed some things, so hopefully a few eyeballs on it can help make it better.

Stuff to note:

1) The quoted items will be lexer terminals, they're in this document for clarity.

2) Items with {} are typed terminals coming from the lexer. They will be read using some pretty simple regexes.

3) Due to the nature of Spin where the number of spaces before a line of code is important, we must supply whitespace to the parser. The actual implementation will have a lot more WS (whitespace) rules everywhere. They're left out here for clarity.

Cheers,

--fletch

UPDATE: The CON parser is definitely weak. Constant expressions should be expanded like the runtime expressions are.

UPDATE2: Yep just remembered 2 language constructs which need to be included. The function call in COGNEW and string(). There's probably more

I'll continue tomorrow.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

Post Edited (dfletch) : 2/26/2008 8:39:33 AM GMT

Phil Pilgrim (PhiPi) · 2008-02-26 09:39

dfletch,

Normally, operator precedence rules are defined by the grammar itself. For example, in this simple grammar, multiplication and division have a higher precedence than addition and subtraction:

<expression> ::= <expression> "+" <factor> | <expression> "-" <factor> | <factor>
<factor> ::= <factor> "*" <term> | <factor> "/" <term> | <term>
<term> ::= <constant> | <variable> | "(" <expression> ")"

In your grammar, all operators seem to have the same precedence. Were you planning to refine this later?

-Phil

dfletch · 2008-02-26 09:59

Phil, yep that's definitely one of the things I've glossed over. Now that you mention it, there's no group expression with parenthesis either!

I'll have a much improved version up tomorrow.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

hippy · 2008-02-26 12:23

Other things not in the grammar so far ...

if / if-else
case
return / abort
quit
in-built functions ( lookup, lookdown, strsize, strcomp etc )
unaries as statements ( var++ , --word[noparse][[/noparse]expr] )
multiple assignments ( var := var := expr, if (var:=expr) == expr )

dfletch · 2008-02-26 16:17

Hippy: Great, thank you! I'm pretty sure I put multiple assignments in there (AssignmentExpression) but the rest you've nailed. I was thiinking of doing unaries as statements by detecting if the expression was at the root. Perhaps I should make them parser constructs instead, huh?

Cheers,

--fletch

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

OwenS · 2008-02-26 17:53

You might want to use Lemon and RE2C, instead of Bison and Flex (respectively). Lemon's grammar is closer to "pure" LALR, and RE2C is faster than Flex. Also, RE2C is definitely UTF-8 clean, and they both don't use global variables.

Did I mention theyre faster?

deSilva · 2008-02-26 18:26

.. and the matter with the different constant contexts (CON, constant(), DAT, before and after multiple allocations,..) is MUCH more complex

dfletch · 2008-02-26 18:38

Owen: Hmm never heard of those. I'll probably stick with Bison/Flex just because I can do this much faster in a familiar environment, but those tools do sound interesting. By the way, with the reentrant option, bison doesn't use globals either! Oh and re: speed... do you really care if compilation takes 1/32 of a second longer because I chose Bison? It's not like we need realtime compiling or something [noparse];)[/noparse]

By the way, we've moved the Spin compiler discussion to another thread. Let's please stick there to keep ontopic.

Cheers,

--fletch

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Join us in the Un-Official Propeller IRC channel: irc.freenode.net #propeller
Newbies, oldies, programmers, math professors, and everyone in-between welcome!
Propeller IRC howto

Paul Baker · 2008-02-27 02:54

How about a Spin to ASM converter that you can feed snippets of spin code and get code equivalent assembly out? People starting out in assembler could get alot of use and understanding out of the utility. The stack can be handled the same as spin, or for additional speedup you could·specify a buffer within the cog.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.

Post Edited (Paul Baker (Parallax)) : 2/27/2008 3:06:17 AM GMT

Javalin · 2008-02-28 16:26

>How about a Spin to ASM converter that you can feed snippets of spin code and get code equivalent assembly out?
shades of the Parallax SX - SX/A and SX/B!

Cool.

Hallomann · 2008-02-28 19:32

Thx!
Should be a sticky topic!^^

Propeller ROM source code HERE

Comments