Standalone, cross-platform Propeller assembler
Cliff L. Biffle
Posts: 206
I've put my reverse engineering documentation to good use, and written an assembler for the P8X32 chips. I am currently using this for actual Propeller development.
www.cliff.biffle.org/software/propeller/propasm/
Overview:
- Runs on any platform with Java available.
- Supports Parallax-format files, including all mnemonics, pseudo-ops (like CALL), and directives (like FIT).
- Command-line tool, usable from scripts or Makefiles.
- Fast.
- Will shortly be open-source (as soon as I finish slapping the GPL on all the files).
Main limitations:
- Does not support SPIN. This is an assembler, not a SPIN compiler.
- Does not yet have a linker -- programs must be in a single file. (You could easily work around this with m4/cpp.)
- There is currently no way to load the output onto the Propeller without using the Propeller Tool. I'll try to fix this shortly.
- Does not yet support expressions (e.g. 80_000_000/9600) in constants or operands. This will also be fixed.
- Does not yet have an Eclipse plugin.
The assembler expects input in "bare" files, devoid of SPIN-style section delimiters. Here's an example that blinks the lights on the demo board once a second:
www.cliff.biffle.org/software/propeller/propasm/
Overview:
- Runs on any platform with Java available.
- Supports Parallax-format files, including all mnemonics, pseudo-ops (like CALL), and directives (like FIT).
- Command-line tool, usable from scripts or Makefiles.
- Fast.
- Will shortly be open-source (as soon as I finish slapping the GPL on all the files).
Main limitations:
- Does not support SPIN. This is an assembler, not a SPIN compiler.
- Does not yet have a linker -- programs must be in a single file. (You could easily work around this with m4/cpp.)
- There is currently no way to load the output onto the Propeller without using the Propeller Tool. I'll try to fix this shortly.
- Does not yet support expressions (e.g. 80_000_000/9600) in constants or operands. This will also be fixed.
- Does not yet have an Eclipse plugin.
The assembler expects input in "bare" files, devoid of SPIN-style section delimiters. Here's an example that blinks the lights on the demo board once a second:
mov DIRA, mask ' outputs
mov time, CNT
add time, cps
blink mov OUTA, mask ' high
waitcnt time, cps
mov OUTA, #0 ' los
waitcnt time, cps
jmp #blink
mask long $00FF_0000 ' light port mask
cps long 80_000_000
time res 1
fit
Comments
You can have access to all 8 cogs even using SPIN to start it all up. COGINIT can start up an assembly program in the same cog that's running the SPIN interpreter, replacing it with the assembly program.
Cliff's assembler packages the assembly program with a tiny SPIN preamble that does just that.
Mike
The primary advantages of propasm over the Parallax assembler are:
- It can run on non-Windows, non-x86 systems.
- It accepts identifiers in non-English character sets.
- It can be scripted from command-line tools.
- It doesn't do SPIN. (Well, I think this is an advantage.
As a cleanroom project, it also doesn't have a lot of the bugs from the official assembler.
More (both the good and the bad) on this page: www.cliff.biffle.org/software/propeller/propasm/differences.html
My point here was to suggest that you build a serial (or USB-to-serial) loader into your assembler, so it would be totally stand-alone. You can look at the PropellerLoader object to see what is required. I'm sorry if we already talked about this and I'm not remembering. Anyway, once you have a loader built into your tool, it could go many places without the Windows hindrance. I imagine this·has got be·your goal, anyway. No·point in·preaching to the choir.
I'm glad you're working on this. I'm sure your tool will turn out nicely and I know it will make the Propeller palatable to a lot of independently-minded people who seek freedom·from Microsoft's hegemony. "May the force be with you, Cliff."
·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chip Gracey
Parallax, Inc.
Post Edited (Chip Gracey (Parallax)) : 10/28/2006 7:37:17 AM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chip Gracey
Parallax, Inc.
You're right -- Propeller Tool does indeed reject out-of-range operands. It even catches out-of-range results of expressions. I'll fix my page!
It does still allow this:
CON x = $100 DAT entry byte $42AA byte $EECC byte x(The bytes wind up being 0xAA, 0xCC, and 0x00, respectively.)
And, yes, I do consider this a bug -- particularly that last case, where it might not even be a typo.
(My assembler has all sorts of interesting bugs as well, but writing a Forth kernel is proving an excellent stress test and I'm flushing them out as I find them.)
I don't disagree, by any means. I'm a big believer in high-level languages -- I am just, as a matter of opinion, not a tremendous fan of SPIN.
I have, however, been using it as a test harness for some of my assembly code, since the turnaround for making changes is much faster (and my compiler doesn't work yet).
By "the shared RAM location" I'm going to assume we mean PAR; correct me if I'm wrong.
Can PAR and the other read-only registers be used in the D field of rdlong/wrlong? I know (from several hours of debugging) that they don't work in the D field of CMP, for example.
Yeah, you read my mind.
I've read the PropellerLoader code; now it's just a question of coercing a traditional UART to speak your variable-length-bit protocol. (Since the start bit gives me one bit-delay of low, it should be straightforward.) Is the Propeller picky about the timing between lows, after the initial sync pattern? The code isn't clear on this.
If it's picky, I'll have to be careful; if not, I can kick the UART into 9-data-bit mode, pack tightly, and see how fast it'll go.
Currently, it's proving more productive to simply finish my Forth, which will give me a Propeller-hosted interactive development environment with debugging. Plugging the board into a TV and keyboard lets me ditch both Windows and Unix.
Edit: Should probably explain: expression support in propasm will let me link the TV driver into my Forth kernel. It relies heavily on compile-time computed constants.
Post Edited (Cliff L. Biffle) : 10/28/2006 8:09:54 AM GMT
Project page: code.google.com/p/propasm/
Source links: code.google.com/p/propasm/source
Issue tracking: code.google.com/p/propasm/issues/list
You can browse the repository by hand, or check out the codebase using Subversion and build it. (You'll need the JDK 5, and either Apache Ant or any modern Java IDE. All other dependencies are included.)
The new features:
- New directive .align allows you to specify an alignment at any point in the source file. The assembler will pad as needed. Example:
- New directive .include allows you to textually include other files (to arbitrary depth).
All my extension directives will start with a dot, since it's syntactically illegal in Parallax's assembler. Wouldn't want to risk colliding with future Parallax extensions.
Example from my Forth kernel:
prompt byte $6F, $6B, $0D .align long ' will pad a byte QUIT .include "quit.paf"Macros coming soon.
Does the assembler automatically pad if the alignment changes like with some byte values followed by a word value?
Does the .include directive allow relative paths?
Thanks, Mike
Yes, but I'm also tempted to make this an optional warning when no explicit .align is present.
Yes. It defaults to the cwd of the assembler (not the location of the source file) and allows both relative paths from there and absolute paths. (This is a bug; it will interpret relative paths relative to the source file in the next rev.)
Windows paths can be written using forward-slashes or doubled backslashes; single backslashes are for escape characters.
Please provide an option for the binary output file to not have the SPIN preamble and a second option for the binary output file to begin at some location other than zero. I would like to be able to have assembly overlays that might load directly into a cog and would like the option to load them somewhere other than the beginning of cog memory without a lot of work.
Thanks, Mike
Yes; I intend to have all optional behavior controlled by directives, and optionally overridden by command-line switches.
They're both on the list. Thanks for the input.
Since the binary format isn't sparse, I'm also considering adding a directive to pad to a particular shared-RAM address. For example, a particular part of my Forth kernel would read
.at $7380 pstack res 64...tentatively speaking.
This would, of course, require code running inside the Cog to process the overlay. I'm all for it, but we'd need to specify the overlay format and a canned overlay loader sequence.
I haven't finished the code. The syntax at the moment is
but I'm open to suggestions.
Currently, they'll come in the same release as expression support.
Thanks. FYI I'm planning on expanding my OS I2C loader to accomodate assembly overlays (since I already have the I2C read/write routines). The routines already just take an EEPROM address, HUB address, and count. I'll be adding option bits for reading and writing from COG memory with an option to jump to the block of data just loaded (and some convention for the return address) and to start a cog using a block of HUB memory just read (without the SPIN interpreter). The loader currently takes under 200 longs, so there's room for at least a 256 long overlay. I'll probably just subdivide a 32K EEPROM space into 15 x 2K or 31 x 1K areas with one area for system overhead. It'll take some real use to see whether something fancier is needed. I'll let you know. Any thoughts are welcome.
Mike
Chip, at the risk of looking dumb I'd have to say that the PropellerLoader code didn't seem to provide me with very much insight into how to actually accomplish the task using a PCs UART and serial drivers. So I had to resort to reversing the communication between the propeller loader and the propeller chip. By recording a "conversation" and then duplicating what the propeller loaders part of the conversation was, I've been able to program a chip from both C# and Python.
My problem now is how to convert any given binary image into a stream of bytes that the propeller's bootload will understand. During the third phase of the programming sequence the body of the program is sent, but looking at what the Propeller Tool is sending this is certainly not just the bytes of the image. If anybody can help me understand how to do this I'd appreciate the help.
Also Chip if you could provide some insight into how you generate and use those LFSR bytestrings I'd appreciate it. When I recorded them they show up as 0xF9, 0xFE, 0xFF,..... and when I send these to the propeller verbatim is certainly responds correctly, but I was unable to duplicate the sequence from the code in the PropellerLoader.
Thanks,
Chad
Planning to release your code? I'd be happy to work with you (though I'm working in Java and Python gives me hives).
You're right, the PropellerLoader -- while technically being "everything we need" -- is not documentation. Give me a flow chart, spec, and timing diagram any day.
What problems are you having with the LFSR? Are you sure your taps are correct?
I'm certainly planning to release my code when I get it working. I'm not particularly a Python junkie myself but its appropriate for my current project (although I guess Java would be too, but I've used Python more) Once the logic of the code works it really should be trivial to get working in either language.
I guess my biggest problem with the LFSR was that I'm not entirely sure what exactly PropellerLoader was doing. I have to admit that I haven't actually run PropellerLoader for myself (has anyone else?), but looking at the code raises more questions than answers. What's the deal with the Echo and where is the serial protocol or any timing at all? Finally I kindof abandoned the idea of try to convert SPIN/Propeller code to real PC code.
I used HDD's Serial Monitor program on my PC to watch the communication between the Propeller Tool and the Propeller at the device driver level. Sure enough it follows the "general" flow of the logic shown in PropellerLoader but seems to use completely different mechanism to achieve it. Here's what I've found out so far.
First the Propeller Tool seems to open the serial port in 115200-8-N-1 mode. After toggling the DTR line to reset the Propeller, it transmits 0xF9 followed by a sequence of 0xFEs and 0xFFs. This sequence is followed by a large block of 0xF9s. I'm sure all this corresponds to the 250 iterations of the LFSR but I don't see how. When I tried to duplicate the LFSR in Python I couldn't get anything even remotely close. Then the Propeller responds with a similar (but different) block of 0xFEs and 0xFFs. Again I'm
sure this corresponds to the LFSR somehow. The Propeller then sends 2 bytes 0xFE 0xFE which I believe is the version number?
After this the Propeller Tools sends a command byte (RAM or EEPROM) followed immediately by block of data which corresponds to the binary image of the program. I haven't spent alot of time analyzing the format of the this data, but it doesn't seem to directly match up with the bytecodes of the program. Something weird is going on here and I think has to do with how the Propeller is "reading" the serial data.
Finally the Propeller Tool sends 0xF9 periodically (every 50msec seems to work) until the Propeller responds with 0xFE. This acknowledgment protocol can be repeated twice more if the command code was programming to the EEPROM.
Of course nothing that I've said is a surprise because the PropellerLoader code made it totally obvious to begin with right [noparse]:)[/noparse]
Currently my implementation is a bit of a hack because I just embedded the LFSR transmit sequences that I recorded directly into my code and send them verbatim. I'd like to actually understand what's going on, but having it work is good enough for now.
I'm also testing with copies of the "program" data that the Propeller Tool sent while loading a chip, but I've proven that the rest of the process is independent of the program code. If I could somehow generate the correct sequence of bytes to send directly from a program binary then we'd be ready to go.
I'm at school right now so I don't have access to my source code or testing files. I'll upload them tonight when I get home for anyone who's interested.
-Chad
The $F9 is the 'calibration pulse'. The start pulse is a short (one) pulse while the two zero bits in the $F9 is a long (zero) pulse. After that, the least significant bit of the LFSR is sent by the Tool, one per character, for 250 bits, with the $FF being a short (one) pulse and the $FE being a long (zero) pulse. Then the Propeller sends back the next 250 bits in the LFSR, also one bit per byte by echoing the $F9 characters, changing them to $F8 to indicate that they were processed properly.
After all the LFSR bits, the Tool and the Propeller shift into a 3 bits per byte mode with each bit cell being 3 bits and the start bit counting as one bit of the first cell (which is always zero). As you might expect, a one is a short (1 zero pulse time) and a zero is a long (2 zero pulse times) and the Propeller echos the data. There's a mix of 8 bit and 32 bit sequences, etc.
You may ask "why this mess?"
I think this protocol is to make sure there's a Propeller there. The Propeller is running off its internal fast clock which is not accurate
enough to do asynchronous serial stuff, so this is a self-clocking system initially (to set up the bit times) and never goes for more than 2 bit times (one data bit) without cleaning up the timing.
Chad, I'm glad you have a protocol analyzer. The dit-dah style protocol was evident from the Spin code, but I wasn't sure how picky the Prop was going to be on bit timing. I'll write up some test code.
Thanks for the info, it really helped everything make alot more sense.
I've been able to implement the LFSR in code rather than sending pre-recorded arrays.
Also I've been able to convert the capture program code back to the original hex code.
Of course this is backwards, but I'm definitely almost there.
-Chad
Post Edited (Chad George) : 10/31/2006 10:26:45 PM GMT
Also attached is a small text file that was used during the chip's development as a 'contract' description between a host (PC) and the Propeller chip. It's terse, but when viewed with the Delphi code, should answer almost any quesion.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chip Gracey
Parallax, Inc.
Rock on. Thanks!
Haven't read a line of Delphi in nearly ten years, but I'll try to make sense of it.
For anyone who's interested I've finished porting Chip's code to Python.
The only non-default library required is pySerial which can be downloaded here
pyserial.sourceforge.net/
Like I said I basically just ported Chip's code directly to Python.
The TalkToHardware() function requires a list of bytes containing the program
to be loaded. I've included Example 1 from the Manual already in the code.
Something like this should send the program to the chip:
TalkToHardware(1, man_ex_1)
Also some people might need to change the ValidCommPorts() routine so it
returns a list of port names that make sense for the OS.
Again, thanks Chip for the documents they helped out immensely.
-Chad
I've verified your loader on Mac OS X 10.4.8, using Python 2.3.5 and the FTDI-supplied serial driver. Nice work.
I disassembled the man_ex_1 byte array, and it's the standard format used by both Propeller Tool and propasm (and documented on my reverse engineering pages). Your loader should work just fine with the output from propasm.
Soon, we will have options.
I knew you didn't like Spin, but...
http://cliffhacks.blogspot.com/2006/10/propeller-spin-object-oriented-my-***.html
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chip Gracey
Parallax, Inc.
That's no so much a reaction to SPIN as to the repeated writeups in the trade press describing it as "object oriented," when it's not. (I do note in that post that the Parallax docs don't use the term.)
However, I do tend to get too fired up about these things, and that was (in hindsight) way harsh. I've taken it down. I'll be reposting soon with a more balanced perspective.
Now it's my turn to ask forgiveness.
I found it harsh too, even though I am not one to say anything at all about what is and is not "object oriented". The one looming factor I see in Spin is the accomplishment of making the whole interpreter fit in 512 longs of one COG. Wow!
I appreciate what you're doing with the standalone assembler (I'm running the Prop Tool on a Mac under VPC). But I still like Spin and the fact that the assembly code is embedded in the Spin, where it is so easy to glue things, objects? methods? devices? algorithms? together.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com
I tend to be pretty picky about the usage of the term OO specifically, due to its general dilution by languages like VB and Python. (I am one of those bitter Smalltalk people.)
I really like one principal concept from Spin, and that's the component model. (Spin calls them objects; I think of them as components.) On a platform with no hardware peripherals, the ability to stitch together soft peripherals in a high-level language is very powerful.
propasm isn't intended to replace that. It's intended for people (like me) who are implementing other, non-Spin languages, or who just really want to get their hands dirty.
At Mike's request, it will soon be able to emit partial object files, intended to be loaded at certain addresses in shared RAM. This will allow propasm and Propeller Tool to live in harmony, as you'll be able to write low-level code using propasm's extended dialect, and then include it in your DAT section with a file directive.